incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lwl <lwl.ro...@gmail.com>
Subject Re: writing speed test
Date Wed, 02 Jun 2010 01:57:02 GMT
MEM: almost 100% (16GB)
-----------------
maybe this is the bottleneck.
writing concerns Memtable and SSTable in memory.

在 2010年6月2日 上午9:48,Shuai Yuan <yuanshuai@supertool.net.cn>写道:

> 在 2010-06-01二的 15:00 -0500,Jonathan Shook写道:
> > Also, what are you meaning specifically by 'slow'? Which measurements
> > are you looking at. What are your baseline constraints for your test
> > system?
> >
> Actually, the problem is the utilizaton of resources(for a single
> machine):
> CPU: 700% / 1600% (16 cores)
> MEM: almost 100% (16GB)
> Swap: almost 0%
> Disk IO(write): 20~30MB / 200MB (7.2k raid5, benchmarked previously)
> NET: up to 100Mbps / 950Mbps (1Gbps, tuned and benchmarked previously)
>
> So the speed of generating load, about 15M/s as reported before seems
> quite slow to me. I assume the system should get at least about 50MB/s
> of Disk IO speed.
>
> MEM? I don't think it plays a major role in this writing game. What's
> the bottleneck of the system?
>
> P.S
> about Consistency Level, I've tried ONE/DCQUORUM and found ONE is about
> 10-15% faster. However that's neither a promising result.
>
> Thanks!
>
> Kevin
> >
> > 2010/6/1 史英杰 <shiyingjie1983@gmail.com>:
> > > Hi, It would be better if we know which Consistency Level did you
> choose,
> > > and what is the schema of test data?
> > >
> > > 在 2010年6月1日 下午4:48,Shuai Yuan <yuanshuai@supertool.net.cn>写道:
> > >>
> > >> Hi all,
> > >>
> > >> I'm testing writing speed of cassandra with 4 servers. I'm confused by
> > >> the behavior of cassandra.
> > >>
> > >> ---env---
> > >> load-data app written in c++, using libcassandra (w/ modified batch
> > >> insert)
> > >> 20 writing threads in 2 processes running on 2 servers
> > >>
> > >> ---optimization---
> > >> 1.turn log level to INFO
> > >> 2.JVM has 8G heap
> > >> 3.32 concurrent read & 128 write in storage-conf.xml, other cache
> > >> enlarged as well.
> > >>
> > >> ---result---
> > >> 1-monitoring by `date;nodetool -h host ring`
> > >> I add all load together and measure the writing speed by
> > >> (load_difference / time_difference), and I get about 15MB/s for the
> > >> whole cluster.
> > >>
> > >> 2-monitoring by `iostat -m 10`
> > >> I can watch the disk_io from the system level and have about 10MB/s -
> > >> 65MB/s for a single machine. Very big variance over time.
> > >>
> > >> 3-monitoring by `iptraf -g`
> > >> In this way I watch the communication between servers and get about
> > >> 10MB/s for a single machine.
> > >>
> > >> ---opinion---
> > >> So, have you checked the writing speed of cassandra? I feel it's quite
> > >> slow currently.
> > >>
> > >> Could anyone confirm this is the normal writing speed of cassandra, or
> > >> please provide someway of improving it?
> > >> --
> > >> Kevin Yuan
> > >> www.yuan-shuai.info
> > >>
> > >>
> > >
> > >
> >
>
> --
> Kevin Yuan
> www.yuan-shuai.info
>
>
>

Mime
View raw message