cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benjamin>
Subject Re: when should new nodes be added to a cluster
Date Mon, 02 Aug 2010 23:48:12 GMT
you have insufficient i/o bandwidth and are seeing reads suffer due to
competition from memtable flushes and compaction.  adding additional
nodes will help some, but i recommend increasing the disk i/o
bandwidth, regardless.


On Mon, Aug 2, 2010 at 11:47 AM, Artie Copeland <> wrote:
> i have a question on what are the signs from cassandra that new nodes should
> be added to the cluster.  We are currently seeing long read times from the
> one node that has about 70GB of data with 60GB in one column family.  we are
> using a replication factor of 3.  I have tracked down the slow to occur when
> either row-read-stage or message-deserializer-pool is high like atleast
> 4000.  my systems are 16core, 3 TB, 48GB mem servers.  we would like to be
> able to use more of the server than just 70GB.
> The system is a realtime system that needs to scale quite large.  Our
> current heap size is 25GB and are getting atleast 50% row cache hit rates.
>  Does it seem strange that cassandra is not able to handle the work load?
>  We perform multislice gets when reading similar to twissandra does.  this
> is to cut down on the network ops.  Looking at iostat it doesnt appear to
> have alot of queued reads.
> What are others seeing when they have to add new nodes?  What data sizes are
> they seeing?  This is needed so we can plan our growth and server purchase
> strategy.
> thanx
> Artie
> --

View raw message