incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lwl <lwl.ro...@gmail.com>
Subject Re: [***SPAM*** ] Re: writing speed test
Date Wed, 02 Jun 2010 02:37:22 GMT
is all the 4 servers' MEM  almost 100%?

在 2010年6月2日 上午10:12,Shuai Yuan <yuanshuai@supertool.net.cn>写道:

> Thanks lwl.
>
> Then is there anyway of tuning this, faster flush to disk or else?
>
> Cheers,
>
> Kevin
>
> 在 2010-06-02三的 09:57 +0800,lwl写道:
> > MEM: almost 100% (16GB)
> > -----------------
> > maybe this is the bottleneck.
> > writing concerns Memtable and SSTable in memory.
> >
> > 在 2010年6月2日 上午9:48,Shuai Yuan <yuanshuai@supertool.net.cn>写
> > 道:
> >         在 2010-06-01二的 15:00 -0500,Jonathan Shook写道:
> >         > Also, what are you meaning specifically by 'slow'? Which
> >         measurements
> >         > are you looking at. What are your baseline constraints for
> >         your test
> >         > system?
> >         >
> >
> >         Actually, the problem is the utilizaton of resources(for a
> >         single
> >         machine):
> >         CPU: 700% / 1600% (16 cores)
> >         MEM: almost 100% (16GB)
> >         Swap: almost 0%
> >         Disk IO(write): 20~30MB / 200MB (7.2k raid5, benchmarked
> >         previously)
> >         NET: up to 100Mbps / 950Mbps (1Gbps, tuned and benchmarked
> >         previously)
> >
> >         So the speed of generating load, about 15M/s as reported
> >         before seems
> >         quite slow to me. I assume the system should get at least
> >         about 50MB/s
> >         of Disk IO speed.
> >
> >         MEM? I don't think it plays a major role in this writing game.
> >         What's
> >         the bottleneck of the system?
> >
> >         P.S
> >         about Consistency Level, I've tried ONE/DCQUORUM and found ONE
> >         is about
> >         10-15% faster. However that's neither a promising result.
> >
> >         Thanks!
> >
> >         Kevin
> >
> >         >
> >         > 2010/6/1 史英杰 <shiyingjie1983@gmail.com>:
> >         > > Hi, It would be better if we know which Consistency Level
> >         did you choose,
> >         > > and what is the schema of test data?
> >         > >
> >         > > 在 2010年6月1日 下午4:48,Shuai Yuan
> >         <yuanshuai@supertool.net.cn>写道:
> >         > >>
> >         > >> Hi all,
> >         > >>
> >         > >> I'm testing writing speed of cassandra with 4 servers.
> >         I'm confused by
> >         > >> the behavior of cassandra.
> >         > >>
> >         > >> ---env---
> >         > >> load-data app written in c++, using libcassandra (w/
> >         modified batch
> >         > >> insert)
> >         > >> 20 writing threads in 2 processes running on 2 servers
> >         > >>
> >         > >> ---optimization---
> >         > >> 1.turn log level to INFO
> >         > >> 2.JVM has 8G heap
> >         > >> 3.32 concurrent read & 128 write in storage-conf.xml,
> >         other cache
> >         > >> enlarged as well.
> >         > >>
> >         > >> ---result---
> >         > >> 1-monitoring by `date;nodetool -h host ring`
> >         > >> I add all load together and measure the writing speed by
> >         > >> (load_difference / time_difference), and I get about
> >         15MB/s for the
> >         > >> whole cluster.
> >         > >>
> >         > >> 2-monitoring by `iostat -m 10`
> >         > >> I can watch the disk_io from the system level and have
> >         about 10MB/s -
> >         > >> 65MB/s for a single machine. Very big variance over time.
> >         > >>
> >         > >> 3-monitoring by `iptraf -g`
> >         > >> In this way I watch the communication between servers and
> >         get about
> >         > >> 10MB/s for a single machine.
> >         > >>
> >         > >> ---opinion---
> >         > >> So, have you checked the writing speed of cassandra? I
> >         feel it's quite
> >         > >> slow currently.
> >         > >>
> >         > >> Could anyone confirm this is the normal writing speed of
> >         cassandra, or
> >         > >> please provide someway of improving it?
> >         > >> --
> >         > >> Kevin Yuan
> >         > >> www.yuan-shuai.info
> >         > >>
> >         > >>
> >         > >
> >         > >
> >         >
> >
> >
> >         --
> >
> >         Kevin Yuan
> >         www.yuan-shuai.info
> >
> >
> >
> >
> >
>
> --
> Shuai Yuan 袁帅
> Supertool Corp. 北京学之途网络科技有限公司
> www.yuan-shuai.info
>
>
>

Mime
View raw message