incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From srmore <comom...@gmail.com>
Subject Re: Cassandra high heap utilization under heavy reads and writes.
Date Wed, 27 Nov 2013 16:58:47 GMT
So, I did a lot of dial turning and heap tuning (came across this nice
writeup about JVM tuning
http://blog.mikiobraun.de/2010/08/cassandra-gc-tuning.html) still no luck
with 1.2.9. I gave up and upgraded to 1.2.12 and since then things are much
much better. I don't run into the heap issue that I used to with 1.2.9.

1.2.12 also looks more stable and seems to work  for us.

-sandeep


On Mon, Nov 25, 2013 at 11:10 AM, Christopher J. Bottaro <
cjbottaro@academicworks.com> wrote:

> Yes, we saw this same behavior.
>
> A couple of months ago, we moved a large portion of our data out of
> Postgres and into Cassandra.  The initial migration was done in a
> "distributed" manner:  we had 600 (or 800, can't remember) processes
> reading from Postgres and writing to Cassandra in tight loops.  This caused
> the exact behavior you described.  We also did a read before a write.
>
> After we got through the initial data migration, our normal workload is
> *much* less writes (and reads for that matter) such that our cluster can
> easily handle it, so we didn't investigate further.
>
> -- C
>
>
> On Sat, Nov 23, 2013 at 10:55 PM, srmore <comomore@gmail.com> wrote:
>
>> Hello,
>> We moved to cassandra 1.2.9 from 1.0.11 to take advantage of the off-heap
>> bloom filters and other improvements.
>>
>> We see a lot of messages dropped under high load conditions. We noticed
>> that when we do heavy read AND write simultaneously (we read first and
>> check whether the key exists if not we write it) Cassandra heap increases
>> dramatically and then gossip marks the node down (as a result of high load
>> on the node).
>>
>>
>> Under heavy 'reads only' we don't see this behavior.  Has anyone seen
>> this behavior ? any suggestions.
>>
>> Thanks !
>>
>>
>>
>

Mime
View raw message