incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Artie Copeland <yeslinux....@gmail.com>
Subject Re: when should new nodes be added to a cluster
Date Tue, 03 Aug 2010 15:55:55 GMT
Thanx for the insight

On Mon, Aug 2, 2010 at 4:48 PM, Benjamin Black <b@b3k.us> wrote:

> you have insufficient i/o bandwidth and are seeing reads suffer due to
> competition from memtable flushes and compaction.  adding additional
> nodes will help some, but i recommend increasing the disk i/o
> bandwidth, regardless.
>
>
> b
>
> On Mon, Aug 2, 2010 at 11:47 AM, Artie Copeland <yeslinux.now@gmail.com>
> wrote:
> > i have a question on what are the signs from cassandra that new nodes
> should
> > be added to the cluster.  We are currently seeing long read times from
> the
> > one node that has about 70GB of data with 60GB in one column family.  we
> are
> > using a replication factor of 3.  I have tracked down the slow to occur
> when
> > either row-read-stage or message-deserializer-pool is high like atleast
> > 4000.  my systems are 16core, 3 TB, 48GB mem servers.  we would like to
> be
> > able to use more of the server than just 70GB.
> > The system is a realtime system that needs to scale quite large.  Our
> > current heap size is 25GB and are getting atleast 50% row cache hit
> rates.
> >  Does it seem strange that cassandra is not able to handle the work load?
> >  We perform multislice gets when reading similar to twissandra does.
>  this
> > is to cut down on the network ops.  Looking at iostat it doesnt appear to
> > have alot of queued reads.
> > What are others seeing when they have to add new nodes?  What data sizes
> are
> > they seeing?  This is needed so we can plan our growth and server
> purchase
> > strategy.
> > thanx
> > Artie
> >
> > --
> > http://yeslinux.org
> > http://yestech.org
> >
>



-- 
http://yeslinux.org
http://yestech.org

Mime
View raw message