incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lwl <lwl.ro...@gmail.com>
Subject Re: [***SPAM*** ] Re: [***SPAM*** ] Re: writing speed test
Date Wed, 02 Jun 2010 03:35:18 GMT
and, why did you set "JVM has 8G heap"?
8g, seems too big.

在 2010年6月2日 上午11:20,lwl <lwl.roger@gmail.com>写道:

> 3.32 concurrent read & 128 write in storage-conf.xml, other cache
> enlarged as well.
> --------------------
>
> maybe you can try to decrease the size of caches.
>
> 在 2010年6月2日 上午11:14,Shuai Yuan <yuanshuai@supertool.net.cn>写道:
>
> 在 2010-06-02三的 10:37 +0800,lwl写道:
>> > is all the 4 servers' MEM  almost 100%?
>>
>> Yes
>>
>> > 在 2010年6月2日 上午10:12,Shuai Yuan <yuanshuai@supertool.net.cn>写
>> > 道:
>> >         Thanks lwl.
>> >
>> >         Then is there anyway of tuning this, faster flush to disk or
>> >         else?
>> >
>> >         Cheers,
>> >
>> >         Kevin
>> >
>> >         在 2010-06-02三的 09:57 +0800,lwl写道:
>> >
>> >         > MEM: almost 100% (16GB)
>> >         > -----------------
>> >         > maybe this is the bottleneck.
>> >         > writing concerns Memtable and SSTable in memory.
>> >         >
>> >         > 在 2010年6月2日 上午9:48,Shuai Yuan
>> >         <yuanshuai@supertool.net.cn>写
>> >         > 道:
>> >         >         在 2010-06-01二的 15:00 -0500,Jonathan Shook写道:
>> >         >         > Also, what are you meaning specifically by 'slow'?
>> >         Which
>> >         >         measurements
>> >         >         > are you looking at. What are your baseline
>> >         constraints for
>> >         >         your test
>> >         >         > system?
>> >         >         >
>> >         >
>> >         >         Actually, the problem is the utilizaton of
>> >         resources(for a
>> >         >         single
>> >         >         machine):
>> >         >         CPU: 700% / 1600% (16 cores)
>> >         >         MEM: almost 100% (16GB)
>> >         >         Swap: almost 0%
>> >         >         Disk IO(write): 20~30MB / 200MB (7.2k raid5,
>> >         benchmarked
>> >         >         previously)
>> >         >         NET: up to 100Mbps / 950Mbps (1Gbps, tuned and
>> >         benchmarked
>> >         >         previously)
>> >         >
>> >         >         So the speed of generating load, about 15M/s as
>> >         reported
>> >         >         before seems
>> >         >         quite slow to me. I assume the system should get at
>> >         least
>> >         >         about 50MB/s
>> >         >         of Disk IO speed.
>> >         >
>> >         >         MEM? I don't think it plays a major role in this
>> >         writing game.
>> >         >         What's
>> >         >         the bottleneck of the system?
>> >         >
>> >         >         P.S
>> >         >         about Consistency Level, I've tried ONE/DCQUORUM and
>> >         found ONE
>> >         >         is about
>> >         >         10-15% faster. However that's neither a promising
>> >         result.
>> >         >
>> >         >         Thanks!
>> >         >
>> >         >         Kevin
>> >         >
>> >         >         >
>> >         >         > 2010/6/1 史英杰 <shiyingjie1983@gmail.com>:
>> >         >         > > Hi, It would be better if we know which
>> >         Consistency Level
>> >         >         did you choose,
>> >         >         > > and what is the schema of test data?
>> >         >         > >
>> >         >         > > 在 2010年6月1日 下午4:48,Shuai Yuan
>> >         >         <yuanshuai@supertool.net.cn>写道:
>> >         >         > >>
>> >         >         > >> Hi all,
>> >         >         > >>
>> >         >         > >> I'm testing writing speed of cassandra with
4
>> >         servers.
>> >         >         I'm confused by
>> >         >         > >> the behavior of cassandra.
>> >         >         > >>
>> >         >         > >> ---env---
>> >         >         > >> load-data app written in c++, using
>> >         libcassandra (w/
>> >         >         modified batch
>> >         >         > >> insert)
>> >         >         > >> 20 writing threads in 2 processes running
on 2
>> >         servers
>> >         >         > >>
>> >         >         > >> ---optimization---
>> >         >         > >> 1.turn log level to INFO
>> >         >         > >> 2.JVM has 8G heap
>> >         >         > >> 3.32 concurrent read & 128 write in
>> >         storage-conf.xml,
>> >         >         other cache
>> >         >         > >> enlarged as well.
>> >         >         > >>
>> >         >         > >> ---result---
>> >         >         > >> 1-monitoring by `date;nodetool -h host ring`
>> >         >         > >> I add all load together and measure the writing
>> >         speed by
>> >         >         > >> (load_difference / time_difference), and
I get
>> >         about
>> >         >         15MB/s for the
>> >         >         > >> whole cluster.
>> >         >         > >>
>> >         >         > >> 2-monitoring by `iostat -m 10`
>> >         >         > >> I can watch the disk_io from the system level
>> >         and have
>> >         >         about 10MB/s -
>> >         >         > >> 65MB/s for a single machine. Very big variance
>> >         over time.
>> >         >         > >>
>> >         >         > >> 3-monitoring by `iptraf -g`
>> >         >         > >> In this way I watch the communication between
>> >         servers and
>> >         >         get about
>> >         >         > >> 10MB/s for a single machine.
>> >         >         > >>
>> >         >         > >> ---opinion---
>> >         >         > >> So, have you checked the writing speed of
>> >         cassandra? I
>> >         >         feel it's quite
>> >         >         > >> slow currently.
>> >         >         > >>
>> >         >         > >> Could anyone confirm this is the normal writing
>> >         speed of
>> >         >         cassandra, or
>> >         >         > >> please provide someway of improving it?
>> >         >         > >> --
>> >         >         > >> Kevin Yuan
>> >         >         > >> www.yuan-shuai.info
>> >         >         > >>
>> >         >         > >>
>> >         >         > >
>> >         >         > >
>> >         >         >
>> >         >
>> >         >
>> >         >         --
>> >         >
>> >         >         Kevin Yuan
>> >         >         www.yuan-shuai.info
>> >         >
>> >         >
>> >         >
>> >         >
>> >         >
>> >
>> >
>> >         --
>> >         Shuai Yuan 袁帅
>> >         Supertool Corp. 北京学之途网络科技有限公司
>> >
>> >         www.yuan-shuai.info
>> >
>> >
>> >
>>
>> --
>> Shuai Yuan 袁帅
>> Supertool Corp. 北京学之途网络科技有限公司
>> www.yuan-shuai.info
>>
>>
>>
>

Mime
View raw message