incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Schuller <peter.schul...@infidyne.com>
Subject Re: High GC activity and OOMs
Date Wed, 09 Nov 2011 21:59:51 GMT
>                Compacted row maximum size: 36904729268

So 36 gigs. As long as you're sure each column is only about 1k, the
total row size should not be a problem.

> While I don't see OOMs when I use only a single thread to page the row, there are lots
of ParNew collections that take about 500ms each and also many full collections.
>
> Do I just not have enough RAM?

Unless something is wrong, I suspect the problem is the total amount
of rows that you have which affects the size of bloom filters. I'm a
bit out of date and don't remember how the memory sizing calculations
are currently made. I suspect you either have to simply add heap size,
or decrease memtable sizing.

I would attach with jconsole and l look at the memory usage after each
concurrent mark sweet phase (or just hit "perform gc" and watch mem
usage). That will tell you the live size. It is likely close to
filling your heap.

I did a quick calculation and each 2 billion column with RF=2 should
result in bloom filter sizes of ~ 3.7 gig per node... I probably am
overlooking something though since you say you have 4 GB of RAM,
unless you have your heap size set to almost 4 gig?

-- 
/ Peter Schuller (@scode, http://worldmodscode.wordpress.com)

Mime
View raw message