incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Christopher J. Bottaro" <cjbott...@academicworks.com>
Subject Re: Cassandra high heap utilization under heavy reads and writes.
Date Mon, 25 Nov 2013 17:10:02 GMT
Yes, we saw this same behavior.

A couple of months ago, we moved a large portion of our data out of
Postgres and into Cassandra.  The initial migration was done in a
"distributed" manner:  we had 600 (or 800, can't remember) processes
reading from Postgres and writing to Cassandra in tight loops.  This caused
the exact behavior you described.  We also did a read before a write.

After we got through the initial data migration, our normal workload is
*much* less writes (and reads for that matter) such that our cluster can
easily handle it, so we didn't investigate further.

-- C


On Sat, Nov 23, 2013 at 10:55 PM, srmore <comomore@gmail.com> wrote:

> Hello,
> We moved to cassandra 1.2.9 from 1.0.11 to take advantage of the off-heap
> bloom filters and other improvements.
>
> We see a lot of messages dropped under high load conditions. We noticed
> that when we do heavy read AND write simultaneously (we read first and
> check whether the key exists if not we write it) Cassandra heap increases
> dramatically and then gossip marks the node down (as a result of high load
> on the node).
>
>
> Under heavy 'reads only' we don't see this behavior.  Has anyone seen this
> behavior ? any suggestions.
>
> Thanks !
>
>
>

Mime
View raw message