incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Baskar Duraikannu" <baskar.duraikannu...@gmail.com>
Subject Re: Performance tests using stress testing tool
Date Thu, 28 Apr 2011 16:06:32 GMT
Thanks Peter. 

When I looked at the benchmark client machine, it was not under any stress in terms of disk
or CPU.  
But test machines are connected through 10/100 mbps switch port (not gigabit). Can this be
a bottleneck?

Thanks
Baskar 


----- Original Message ----- 
From: Peter Schuller 
To: user@cassandra.apache.org 
Sent: Thursday, April 28, 2011 2:34 AM
Subject: Re: Performance tests using stress testing tool


> a) I am not seeing cpu usage more than 10pct.

Sounds like the benchmarking client is bottlenecking.

> In some of the forums, i see
> that 8 cpu 32 gb is considered as good sweet spot for cassandra. Is this
> true?

Seems reasonable in a very general sense, but of course varies with use-case.

> Also when would I see real cpu spikes. At this moment looks like 4
> core is more than sufficient.

In general, the more requests and columns you read and write, the more
you'll be bottlenecking on CPU. The larger individual columns (and
thus fewer columns), the more you'll be bound on disk instead.

In your case the bottleneck seems to be the benchmark I think.

> B) iostat -x is reporting avgqueue size of around 0.25 and await time of
> around 30 ms. What would be acceptable queue size and await time?

Any avgqueuesize significantly below 1 is generally good. For close to
1 or higher than one, it will depend on your access pattern, latency
demands, and the nature of your storage device (e.g., SSD:s, RAID:s
can sustain concurrent I/O).

To simplify, there is some maximum amount of I/O requests that your
storage device will service concurrently. For a normal disk, this is 1
request (I'm ignoring optimizations due to TCQ/NCQ which can be
significant sometimes). As long as you're below the saturation point
it's mostly about statistics and varying I/O patterns causing latency.
The less saturated you are, the better your average latency will be.

Once you're *above* saturation latency goes haywire as you don't
service as many I/O requests as are coming in.

There is a grey area in between where latency will be very sensitive
to smallish changes in I/O load but aggregate throughput remaining
below what can be sustained.

-- 
/ Peter Schuller

Mime
View raw message