incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Ellis <jbel...@gmail.com>
Subject Re: TimedOutException
Date Wed, 16 Dec 2009 20:07:14 GMT
On Wed, Dec 16, 2009 at 12:46 PM, Ramzi Rabah <rrabah@playdom.com> wrote:
> We are observing increasing number of TimedOutExceptions in cassandra
> 0.5 trunk although the load seems fairly low (about 400 reads/writes
> per second).
> cfstats reports that operations are taking less than 2 ms on average.
>
> 2 Things I have noticed looking at the source code.
>
> 1- TimedOutExceptions are silently swallowed by Cassandra and not
> reported in the logs even at debug level

It's reported to the client.  Hardly "swallowed" :)

> 2- readstats does not account for these long time running queries that
> time out.

Right.  But the CF-level stats do.

> I'm wondering, what could be causing the system to go haywire like
> this?

Hard to say without more information.  One shot in the dark is that
get_key_range is a major offender sometimes, as well as workloads that
do lots of deletes + re-inserts for the same keys.

-Jonathan

Mime
View raw message