cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ben Bromhead <>
Subject Re: Unexplained query slowness
Date Wed, 25 Feb 2015 22:58:34 GMT
Have a look at your column family histograms (nodetool cfhistograms iirc),
if you notice things like a very long tail, a double hump or outliers it
would indicate something wrong with your data model or you have a hot
partition key/s.

Also looking at your 99 and 95 percentile latencies will just hide these
occasional high latency reads as they fall outside these percentiles.

If you are running a stock config, first rule out that its not your data
model, then investigate things like disk latency, noisy neighbours (if you
are on vms/ in the cloud).

On 26 February 2015 at 03:01, Marcelo Valle (BLOOMBERG/ LONDON) <> wrote:

> I am sorry if it's too basic and you already looked at that, but the first
> thing I would ask would be the data model.
> What data model are you using (how is your data partitioned)? What queries
> are you running? If you are using ALLOW FILTERING, for instance, it will be
> very easy to say why it's slow.
> Most times people get slow queries in Cassandra they are using the wrong
> data model.
> []s
> From:
> Subject: Re:Unexplained query slowness
> Our Cassandra database just rolled to live last night. I’m looking at our
> query performance, and overall it is very good, but perhaps 1 in 10,000
> queries takes several hundred milliseconds (up to a full second). I’ve
> grepped for GC in the system.log on all nodes, and there aren’t any recent
> GC events. I’m executing ~500 queries per second, which produces negligible
> load and CPU utilization. I have very minimal writes (one every few
> minutes). The slow queries are across the board. There isn’t one particular
> query that is slow.
> I’m running 2.0.12 with SSD’s. I’ve got a 10 node cluster with RF=3.
> I have no idea where to even begin to look. Any thoughts on where to start
> would be greatly appreciated.
> Robert


Ben Bromhead

Instaclustr | | @instaclustr
<> | (650) 284 9692

View raw message