cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tyler Hobbs <ty...@datastax.com>
Subject Re: node vs node latency
Date Sun, 08 Jul 2012 02:51:22 GMT
Those latencies look like the difference between a couple of disk seeks and
reading something that's already in the os cache.

The dynamic snitch will favor nodes with lower latencies.  Once a node has
served enough reads, it might not have to hit disk very often, which
produces lower latencies.  So, if you have a hot dataset that fits into
memory, the dynamic snitch starts a positive feedback loop where most reads
will be served from one replica.

I'm guessing the node with the low latencies is serving most of your reads.
You can look at how quickly the total read count is increasing for each of
the replicas to confirm this.  It's not easy to do with only nodetool
cfstats, but something like OpsCenter would help.

On Fri, Jul 6, 2012 at 9:11 PM, Deno Vichas <deno@syncopated.net> wrote:

> all,
>
> what would explain a huge different (12ms vs 0.1ms) in read latency from
> node to node.  i've got a 4 node cluster w/ replication factor of 3 using
> hector.  i'm seeing these numbers with nodetool cfstats.
>
>
> thx,
> deno
>



-- 
Tyler Hobbs
DataStax <http://datastax.com/>

Mime
View raw message