incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Roland Gude <roland.g...@yoochoose.com>
Subject AW: Garbage collection freezes cassandra node
Date Mon, 19 Dec 2011 16:37:31 GMT
Tuning garbage colletion is really hard. Espescially if you do not know why garbage collection
stalls.
In general I must say I have never seen a software which shipped with such a good garbage
collection configuration as Cassandra.

The thing that looks suspiscious is that the major collections appear regularly in 1 hour
intervals.
The only software I know of is RMI explicit collections (which I am not certain Cassandra
uses)

You could avoid those with
-XX:+DisableExplicitGC
In the Cassandra start script

But I assume that Cassandra itself makes use of explicit gc  as well so this might have some
nasty side effects.
Unless someone tells you never to do that on Cassandra I would go for a try and see what happens.
Keep flushes and deletions of sstables on your mind though, they are somehow tied to gc I
think.

Another option (if it is RMI) is to make the RMI full gc threshold larger for timeouts to
occur less often.
-Dsun.rmi.dgc.client.gcInterval=600000
-Dsun.rmi.dgc.server.gcInterval=600000

The number is ms until a gc is triggered.

Anyway.. same warning as before applies

Cheers.



Von: Rene Kochen [mailto:Rene.Kochen@emea.schange.com]
Gesendet: Montag, 19. Dezember 2011 16:35
An: user@cassandra.apache.org
Betreff: Garbage collection freezes cassandra node

I recently see the following garbage collect behavior in our performance tests (the attached
chart shows the heap-size in MB):

[cid:image001.jpg@01CCBE73.8B567F80]

During the garbage collections, Cassandra freezes for about ten seconds. I observe the following
log entries:

"GC for ConcurrentMarkSweep: 11597 ms for 1 collections, 1887933144 used; max is 8550678528"

I use Windows Server 2008 and Cassandra 0.7.10 with min and max heap size set to 8 GB.

What can I do to make Cassandra not freeze? Just allocate more memory?

Thanks,

Rene
Mime
View raw message