cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pranay akula <pranay.akula2...@gmail.com>
Subject Re: High load on few nodes in a DC.
Date Thu, 29 Sep 2016 15:58:26 GMT
Yes we are using token aware but not shuffling replicas.

On Wed, Sep 21, 2016 at 10:04 AM, Romain Hardouin <romainh_ml@yahoo.fr>
wrote:

> Hi,
>
> Do you shuffle the replicas with TokenAwarePolicy?
> TokenAwarePolicy(LoadBalancingPolicy childPolicy, boolean
> shuffleReplicas)
>
> Best,
>
> Romain
> Le Mardi 20 septembre 2016 15h47, Pranay akula <pranay.akula2323@gmail.com>
> a écrit :
>
>
> I was a able to find the hotspots causing the load,but the size of these
> partitions are in KB and no tombstones and no.of sstables is only 2 what
> else i need to debug to find the reason for high load for some nodes.
>   we are also using unlogged batches is that can be the reason ?? how to
> find which node is serving as a coordinator for un logged batches?? we are
> using token-aware policy.
>
> thanks
>
>
>
> On Mon, Sep 19, 2016 at 12:29 PM, Pranay akula <pranay.akula2323@gmail.com
> > wrote:
>
> I was able to see most used partitions but the nodes with less load are
> serving more read and write requests for that particular partitions when
> compared to nodes with high load, how can i find if these nodes are serving
> as co-coordinators for those read and write requests ?? how can i find the
> token range for these particular partitions and which node is the primary
> for these partition ??
>
>
> Thanks
>
> On Mon, Sep 19, 2016 at 11:04 AM, Pranay akula <pranay.akula2323@gmail.com
> > wrote:
>
> Hai Jeff,
>
> Thank, we are using RF 3 and cassandra version 2.1.8.
>
> Thanks
> Pranay.
>
> On Mon, Sep 19, 2016 at 10:55 AM, Jeff Jirsa <jeff.jirsa@crowdstrike.com>
> wrote:
>
> Is your replication_factor 2? Or is it 3?  What version are you using?
>
> The most likely answer is some individual partition that’s either being
> written/read more than others, or is somehow impacting the cluster (wide
> rows are a natural candidate).
>
> You don’t mention your version, but most modern versions of Cassandra ship
> with ‘nodetool toppartitions’, which will help you identify frequently
> written/read partitions – perhaps you can use that to identify a hotspot
> due to some external behavior (some partition being read thousands of
> times, over and over could certainly drive up load).
>
> -          Jeff
>
> *From: *Pranay akula <pranay.akula2323@gmail.com>
> *Reply-To: *"user@cassandra.apache.org" <user@cassandra.apache.org>
> *Date: *Monday, September 19, 2016 at 7:53 AM
> *To: *"user@cassandra.apache.org" <user@cassandra.apache.org>
> *Subject: *High load on few nodes in a DC.
>
> when our cluster was under load  i am seeing  1 or 2 nodes are on more
> load consistently when compared to others in dc i am not seeing any GC
> pauses or wide partitions  is this can be those nodes are continuously
> serving as coordinators ?? how can  i find what is the reason for high load
> on those two nodes ?? We are using Vnode.
>
>
> Thanks
> Pranay.
>
>
>
>
>
>
>

Mime
View raw message