incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David McNelis <dmcne...@gmail.com>
Subject Re:
Date Wed, 12 Mar 2014 23:38:03 GMT
Not knowing anything about your data structure (to expand on what Edward
said), you could be running into something where you've got some hot keys
that are getting the majority of writes during those heavily loads.... more
specifically I might look for a single key that you're writing, since
you're RF=3 and you have 3 nodes specifically that are causing problems.



On Wed, Mar 12, 2014 at 7:28 PM, Russ Bradberry <rbradberry@gmail.com>wrote:

> I wouldn't go above 8G unless you have a very powerful machine that can
> keep the GC pauses low.
>
> Sent from my iPhone
>
> On Mar 12, 2014, at 7:11 PM, Edward Capriolo <edlinuxguru@gmail.com>
> wrote:
>
> That is too much ram for cassandra make that 6g to 10g.
>
> The uneven perf could be because your requests do not shard evenly.
>
> On Wednesday, March 12, 2014, Batranut Bogdan <batranub@yahoo.com> wrote:
> > Hello all,
> >
> > The environment:
> >
> > I have a 6 node Cassandra cluster. On each node I have:
> > - 32 G RAM
> > - 24 G RAM for cassa
> > - ~150 - 200 MB/s disk speed
> > - tomcat 6 with axis2 webservice that uses the datastax java driver to
> make
> > asynch reads / writes
> > - replication factor for the keyspace is 3
> >
> > All nodes in the same data center
> > The clients that read / write are in the same datacenter so network is
> > Gigabit.
> >
> > Writes are performed via exposed methods from Axis2 WS . The Cassandra
> Java
> > driver uses the round robin load balancing policy so all the nodes in the
> > cluster should be hit with write requests under heavy write or read load
> > from multiple clients.
> >
> > I am monitoring all nodes with JConsole from another box.
> >
> > The problem:
> >
> > When wrinting to a particular column family, only 3 nodes have high CPU
> load
> > ~ 80 - 99 %. The remaining 3 are at ~2 - 10 % CPU. During writes, reads
> > timeout.
> >
> > I need more speed for both writes of reads. Due to the fact that 3 nodes
> > barely have CPU activity leads me to think that the whole potential for
> C*
> > is not touched.
> >
> > I am running out of ideas...
> >
> > If further details about the environment I can provide them.
> >
> >
> > Thank you very much.
>
> --
> Sorry this was sent from mobile. Will do less grammar and spell check than
> usual.
>
>

Mime
View raw message