cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shuai Yuan <yuansh...@supertool.net.cn>
Subject Re: [***SPAM*** ] Re: [***SPAM*** ] Re: [***SPAM*** ] Re: [***SPAM*** ] Re: writing speed test
Date Wed, 02 Jun 2010 08:34:16 GMT
Still seems MEM.

However it's hard to convince that constantly writing(even great amount
of data) needs so much MEM(16GB). The process is quite simple,

input_data -> memtable -> flush to disk

right? What does cassandra need so much MEM for?

Thanks!


在 2010-06-02三的 16:24 +0800,lwl写道:
> No.
> But I did some capacity tests about another distributed system.
> Your former test cost too much MEM, it was the bottleneck.
> caches and JVM cost MEM, so I suggested to decrease them.
> 
> 
> What is the bottleneck of your current test now?
> 
> 
> 在 2010年6月2日 下午4:13,Shuai Yuan <yuanshuai@supertool.net.cn>写
> 道:
>         Hi,
>         
>         I tried,
>         
>         1-consistency level ZERO
>         
>         2-JVM heap 4GB
>         
>         3-normal Memtable cache
>         
>         and now I have about 30% improvment.
>         
>         However I want to know if you have also done w/r benchmark and
>         what's
>         the result?
>         
>         在 2010-06-02三的 11:35 +0800,lwl写道:
>         
>         > and, why did you set "JVM has 8G heap"?
>         > 8g, seems too big.
>         >
>         > 在 2010年6月2日 上午11:20,lwl <lwl.roger@gmail.com>写道:
>         >         3.32 concurrent read & 128 write in
>         storage-conf.xml, other
>         >         cache
>         >         enlarged as well.
>         >         --------------------
>         >
>         >
>         >         maybe you can try to decrease the size of caches.
>         >
>         >         在 2010年6月2日 上午11:14,Shuai Yuan
>         >         <yuanshuai@supertool.net.cn>写道:
>         >
>         >
>         >                 在 2010-06-02三的 10:37 +0800,lwl写道:
>         >                 > is all the 4 servers' MEM  almost 100%?
>         >
>         >
>         >                 Yes
>         >
>         >
>         >                 > 在 2010年6月2日 上午10:12,Shuai Yuan
>         >                 <yuanshuai@supertool.net.cn>写
>         >                 > 道:
>         >                 >         Thanks lwl.
>         >                 >
>         >                 >         Then is there anyway of tuning
>         this, faster
>         >                 flush to disk or
>         >                 >         else?
>         >                 >
>         >                 >         Cheers,
>         >                 >
>         >                 >         Kevin
>         >                 >
>         >                 >         在 2010-06-02三的 09:57 +0800,lwl
>         写道:
>         >                 >
>         >                 >         > MEM: almost 100% (16GB)
>         >                 >         > -----------------
>         >                 >         > maybe this is the bottleneck.
>         >                 >         > writing concerns Memtable and
>         SSTable in
>         >                 memory.
>         >                 >         >
>         >                 >         > 在 2010年6月2日 上午9:48,Shuai
>         Yuan
>         >                 >         <yuanshuai@supertool.net.cn>写
>         >                 >         > 道:
>         >                 >         >         在 2010-06-01二的 15:00
>         -0500,
>         >                 Jonathan Shook写道:
>         >                 >         >         > Also, what are you
>         meaning
>         >                 specifically by 'slow'?
>         >                 >         Which
>         >                 >         >         measurements
>         >                 >         >         > are you looking at.
>         What are
>         >                 your baseline
>         >                 >         constraints for
>         >                 >         >         your test
>         >                 >         >         > system?
>         >                 >         >         >
>         >                 >         >
>         >                 >         >         Actually, the problem is
>         the
>         >                 utilizaton of
>         >                 >         resources(for a
>         >                 >         >         single
>         >                 >         >         machine):
>         >                 >         >         CPU: 700% / 1600% (16
>         cores)
>         >                 >         >         MEM: almost 100% (16GB)
>         >                 >         >         Swap: almost 0%
>         >                 >         >         Disk IO(write):
>         20~30MB / 200MB
>         >                 (7.2k raid5,
>         >                 >         benchmarked
>         >                 >         >         previously)
>         >                 >         >         NET: up to 100Mbps /
>         950Mbps
>         >                 (1Gbps, tuned and
>         >                 >         benchmarked
>         >                 >         >         previously)
>         >                 >         >
>         >                 >         >         So the speed of
>         generating load,
>         >                 about 15M/s as
>         >                 >         reported
>         >                 >         >         before seems
>         >                 >         >         quite slow to me. I
>         assume the
>         >                 system should get at
>         >                 >         least
>         >                 >         >         about 50MB/s
>         >                 >         >         of Disk IO speed.
>         >                 >         >
>         >                 >         >         MEM? I don't think it
>         plays a
>         >                 major role in this
>         >                 >         writing game.
>         >                 >         >         What's
>         >                 >         >         the bottleneck of the
>         system?
>         >                 >         >
>         >                 >         >         P.S
>         >                 >         >         about Consistency Level,
>         I've
>         >                 tried ONE/DCQUORUM and
>         >                 >         found ONE
>         >                 >         >         is about
>         >                 >         >         10-15% faster. However
>         that's
>         >                 neither a promising
>         >                 >         result.
>         >                 >         >
>         >                 >         >         Thanks!
>         >                 >         >
>         >                 >         >         Kevin
>         >                 >         >
>         >                 >         >         >
>         >                 >         >         > 2010/6/1 史英杰
>         >                 <shiyingjie1983@gmail.com>:
>         >                 >         >         > > Hi, It would be
>         better if we
>         >                 know which
>         >                 >         Consistency Level
>         >                 >         >         did you choose,
>         >                 >         >         > > and what is the
>         schema of test
>         >                 data?
>         >                 >         >         > >
>         >                 >         >         > > 在 2010年6月1日
下午
>         4:48,
>         >                 Shuai Yuan
>         >                 >         >
>         <yuanshuai@supertool.net.cn>写道:
>         >                 >         >         > >>
>         >                 >         >         > >> Hi all,
>         >                 >         >         > >>
>         >                 >         >         > >> I'm testing writing
>         speed of
>         >                 cassandra with 4
>         >                 >         servers.
>         >                 >         >         I'm confused by
>         >                 >         >         > >> the behavior of
>         cassandra.
>         >                 >         >         > >>
>         >                 >         >         > >> ---env---
>         >                 >         >         > >> load-data app
>         written in c++,
>         >                 using
>         >                 >         libcassandra (w/
>         >                 >         >         modified batch
>         >                 >         >         > >> insert)
>         >                 >         >         > >> 20 writing threads
>         in 2
>         >                 processes running on 2
>         >                 >         servers
>         >                 >         >         > >>
>         >                 >         >         > >> ---optimization---
>         >                 >         >         > >> 1.turn log level
to
>         INFO
>         >                 >         >         > >> 2.JVM has 8G heap
>         >                 >         >         > >> 3.32 concurrent
>         read & 128
>         >                 write in
>         >                 >         storage-conf.xml,
>         >                 >         >         other cache
>         >                 >         >         > >> enlarged as well.
>         >                 >         >         > >>
>         >                 >         >         > >> ---result---
>         >                 >         >         > >> 1-monitoring by
>         >                 `date;nodetool -h host ring`
>         >                 >         >         > >> I add all load
>         together and
>         >                 measure the writing
>         >                 >         speed by
>         >                 >         >         > >> (load_difference
/
>         >                 time_difference), and I get
>         >                 >         about
>         >                 >         >         15MB/s for the
>         >                 >         >         > >> whole cluster.
>         >                 >         >         > >>
>         >                 >         >         > >> 2-monitoring by
>         `iostat -m
>         >                 10`
>         >                 >         >         > >> I can watch the
>         disk_io from
>         >                 the system level
>         >                 >         and have
>         >                 >         >         about 10MB/s -
>         >                 >         >         > >> 65MB/s for a single
>         machine.
>         >                 Very big variance
>         >                 >         over time.
>         >                 >         >         > >>
>         >                 >         >         > >> 3-monitoring by
>         `iptraf -g`
>         >                 >         >         > >> In this way I watch
>         the
>         >                 communication between
>         >                 >         servers and
>         >                 >         >         get about
>         >                 >         >         > >> 10MB/s for a single
>         machine.
>         >                 >         >         > >>
>         >                 >         >         > >> ---opinion---
>         >                 >         >         > >> So, have you
>         checked the
>         >                 writing speed of
>         >                 >         cassandra? I
>         >                 >         >         feel it's quite
>         >                 >         >         > >> slow currently.
>         >                 >         >         > >>
>         >                 >         >         > >> Could anyone
>         confirm this is
>         >                 the normal writing
>         >                 >         speed of
>         >                 >         >         cassandra, or
>         >                 >         >         > >> please provide
>         someway of
>         >                 improving it?
>         >                 >         >         > >> --
>         >                 >         >         > >> Kevin Yuan
>         >                 >         >         > >> www.yuan-shuai.info
>         >                 >         >         > >>
>         >                 >         >         > >>
>         >                 >         >         > >
>         >                 >         >         > >
>         >                 >         >         >
>         >                 >         >
>         >                 >         >
>         >                 >         >         --
>         >                 >         >
>         >                 >         >         Kevin Yuan
>         >                 >         >         www.yuan-shuai.info
>         >                 >         >
>         >                 >         >
>         >                 >         >
>         >                 >         >
>         >                 >         >
>         >                 >
>         >                 >
>         >                 >         --
>         >                 >         Shuai Yuan 袁帅
>         >                 >         Supertool Corp. 北京学之途网络科技
>         有限公司
>         >                 >
>         >                 >         www.yuan-shuai.info
>         >                 >
>         >                 >
>         >                 >
>         >
>         >
>         >                 --
>         >
>         >                 Shuai Yuan 袁帅
>         >                 Supertool Corp. 北京学之途网络科技有限公司
>         >                 www.yuan-shuai.info
>         >
>         >
>         >
>         >
>         >
>         >
>         >
>         
>         
>         --
>         
>         Shuai Yuan 袁帅
>         Supertool Corp. 北京学之途网络科技有限公司
>         www.yuan-shuai.info
>         
>         
>         
> 
> 

-- 
Kevin Yuan
www.yuan-shuai.info



Mime
View raw message