incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Philippe <watche...@gmail.com>
Subject What does a cluster throttled by the network look like ?
Date Sun, 30 Oct 2011 22:14:31 GMT
Dear all,
I'm working with a 12-node, RF=3 cluster on low-end hardware (core i5 with
16GB of RAM & SATA disks).
I'm using a BOP and each node has a load between 50GB and 100GB (yes, I
apparently did not set my tokens right... I'll fix that later).

I'm hitting the cluster with a little over 100 concurrent threads during
reads & writes in counter columns & plain columns. When I peek at my
various servers & look at my munin graphs, it looks like everything is
great : iostat -dmx shows peeks of 5% utilization at worst and vmstat shows
peeks of 30% CPU utilization, no swapping and plenty of caching. Munin
shows me that the JVM's heap is prancing around 50% of its allocated amount
(and I have no row caching for now).

So my thinking has been that the cluster is under-utilized but when I
increase the number of threads hitting the cluster, I do not see the
throughput increase which has been puzzling. I do see an occasional
TimedOutException now that I've made my batches smaller (to dozens of
mutations rather than hundreds) and my slices smaller too.

What I do see is that each server in the cluster has about 60-80Mb/s
traffic in & out and I'm running on 100Mb ethernet. I seem to remember that
you can only get to a little over 80% of the available ethernet bandwidth
in real-life.
Could my performance be limited by the Network Adapters ?

Thanks

Mime
View raw message