cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jivko donev <jivko_...@yahoo.com>
Subject Re: OOM(Java heap space) on start-up during commit log replaying
Date Wed, 13 Aug 2014 11:23:07 GMT
Graham,
Thanks for the reply. As I stated in mine first mail increasing the heap size fixes the problem
but I'm more interesting in figuring out the right properties for commitlog and memtable sizes
when we need to keep the heap smaller. 
Also I think we are not seeing CASSANDRA-7546 as I apply your patch but the problem still
persists. 
What more details do you need? I'll be happy to provide them.


On Wednesday, August 13, 2014 1:05 AM, graham sanderson <graham@vast.com> wrote:
 


Agreed need more details; and just start by increasing heap because that may wells solve the
problem.

I have just observed (which makes sense when you think about it) while testing fix for https://issues.apache.org/jira/browse/CASSANDRA-7546,
that if you are replaying a commit log which has a high level of updates for the same partition
key, you can hit that issue - excess memory allocation under high contention for the same
partition key - (this might not cause OOM but will certainly massively tax GC and it sounds
like you don’t have a lot/any headroom).

On Aug 12, 2014, at 12:31 PM, Robert Coli <rcoli@eventbrite.com> wrote:


>
>On Tue, Aug 12, 2014 at 9:34 AM, jivko donev <jivko_d88@yahoo.com> wrote:
>
>We have a node with commit log director ~4G. During start-up of the node on commit log
replaying the used heap space is constantly growing ending with OOM error. 
>>
>>
>>
>>The heap size and new heap size properties are - 1G and 256M. We are using the default
settings for commitlog_sync, commitlog_sync_period_in_ms and commitlog_segment_size_in_mb.
>
>
>What version of Cassandra?
>
>
>1G is tiny for cassandra heap. There is a direct relationship between the data in the
commitlog and memtables and in the heap. You almost certainly need more heap or less commitlog.
>
>
>=Rob
>  
Mime
View raw message