cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ben Kaehne <ben.kae...@sirca.org.au>
Subject Re: Cassandra nodes loaded unequally
Date Thu, 18 Oct 2012 22:04:10 GMT
After some time. I believe this is correct. The "load" seems to be
correlated to compactions/number of files for keyspace/IO etc.

Thanks all!

Regards,

On Thu, Oct 18, 2012 at 9:35 PM, aaron morton <aaron@thelastpickle.com>wrote:

> At times of high load check the CPU % for the java service running C* to
> confirm C* is the source of load.
>
> If the load is generated from C* check the logs (or use OpsCentre / other
> monitoring) to see if it correlated to compaction, or Garbage Collection or
> repair or high throughput.
>
> Cheers
>
> -----------------
> Aaron Morton
> Freelance Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 17/10/2012, at 12:22 PM, Ben Kaehne <ben.kaehne@sirca.org.au> wrote:
>
> Nothing unusual.
>
> All servers are exactly the same. Nothing unusual in the log files. Is
> there any level of logging that I should be turning on?
>
> Regards,
>
> On Wed, Oct 17, 2012 at 9:51 AM, Andrey Ilinykh <ailinykh@gmail.com>wrote:
>
>> With your environment (3 nodes, RF=3) it is very difficult to get
>> uneven load. Each node receives the same number of read/write
>> requests. Probably something is wrong on low level, OS or VM. Do you
>> see anything unusual in log files?
>>
>> Andrey
>>
>> On Tue, Oct 16, 2012 at 3:40 PM, Ben Kaehne <ben.kaehne@sirca.org.au>
>> wrote:
>> > Not connecting to the same node every time. Using Hector to ensure an
>> even
>> > distribution of connections accross the cluster.
>> >
>> > Regards,
>> >
>> > On Sat, Oct 13, 2012 at 4:15 AM, B. Todd Burruss <btoddb@gmail.com>
>> wrote:
>> >>
>> >> are you connecting to the same node every time?  if so, spread out
>> >> your connections across the ring
>> >>
>> >> On Fri, Oct 12, 2012 at 1:22 AM, Alexey Zotov <azotov@griddynamics.com
>> >
>> >> wrote:
>> >> > Hi Ben,
>> >> >
>> >> > I suggest you to compare amount of queries for each node. May be the
>> >> > problem
>> >> > is on the client side.
>> >> > Yoy can do that using JMX:
>> >> > "org.apache.cassandra.db:type=ColumnFamilies,keyspace=<YOUR
>> >> > KEYSPACE>,columnfamily=<YOUR CF>","ReadCount"
>> >> > "org.apache.cassandra.db:type=ColumnFamilies,keyspace=<YOUR
>> >> > KEYSPACE>,columnfamily=<YOUR CF>","WriteCount"
>> >> >
>> >> > Also I suggest to check output of "nodetool compactionstats".
>> >> >
>> >> > --
>> >> > Alexey
>> >> >
>> >> >
>> >
>> >
>> >
>> >
>> > --
>> > -Ben
>>
>
>
>
> --
> -Ben
>
>
>


-- 
-Ben

Mime
View raw message