incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Viktor Jevdokimov <Viktor.Jevdoki...@adform.com>
Subject RE: Tools for analize cassandra gc.log
Date Wed, 04 Jul 2012 14:02:18 GMT
All of written below is my personal experience, as goals I wanted to achieve.

We use these parameters in PROD for GC logs:
JVM_OPTS="$JVM_OPTS -XX:+PrintGCTimeStamps"
JVM_OPTS="$JVM_OPTS -XX:+PrintGCDetails"
JVM_OPTS="$JVM_OPTS -Xloggc:/var/log/cassandra/gc-`date +%s`.log"

Log rotation works from Oracle Java 1.7.0_02:
JVM_OPTS="$JVM_OPTS -XX:+UseGCLogFileRotation"
JVM_OPTS="$JVM_OPTS -XX:NumberOfGCLogFiles=10"
JVM_OPTS="$JVM_OPTS -XX:GCLogFileSize=10M"

Adding other parameters may generate log not readable by many/any analyzer.
Adding JVM_OPTS="$JVM_OPTS -XX:+PrintHeapAtGC" will not break analizers below, but useful
only to review manually.


1. GCHisto  http://java.net/projects/gchisto/ - to load different logs and compare some numbers
(pauses and number of GC's). Very useful for comparison.
2. GCViewer http://www.tagtraum.com/gcviewer.html - shows memory consumption and pauses stats
and graphs.

3. PrintGCStats http://java.net/projects/printgcstats/ - text only, Linux only.
4. Garbagecat - a garbage, very little info, text only, while GCHisto & GCViewer much
more info and graphical.

Considering what to play with, understand Java GC first:
http://www.scribd.com/doc/37127094/GCTuningPresentationFISL10
http://developers.sun.com/mobility/midp/articles/garbagecollection2/
http://www.oracle.com/technetwork/java/javase/gc-tuning-6-140523.html

Most objects taking time in GC pause: Memtables, Caches; other objects dying fast - cleaned,
instead of copied/promoted. So play with memtable size also, cache invalidation.

GC frequency will grow when compaction is running, but not pauses (very short living objects).

Testing a lot of scenarios we found standard Cassandra GC settings are fine for most cases,
you may need to play with max heap and young heap size. Other than that you may play with:
-XX:+UseNUMA
-XX:SurvivorRatio=8 (4-16)

Not worth to play with:
-XX:MaxTenuringThreshold=1 (0 or more than 1 is not a good thing)

So mostly changing standard settings is a subject of achievement. Tuning GC is a last resort
to gain something.





Best regards / Pagarbiai

Viktor Jevdokimov
Senior Developer

Email: Viktor.Jevdokimov@adform.com
Phone: +370 5 212 3063
Fax: +370 5 261 0453

J. Jasinskio 16C,
LT-01112 Vilnius,
Lithuania



Disclaimer: The information contained in this message and attachments is intended solely for
the attention and use of the named addressee and may be confidential. If you are not the intended
recipient, you are reminded that the information remains the property of the sender. You must
not use, disclose, distribute, copy, print or rely on this e-mail. If you have received this
message in error, please contact the sender immediately and irrevocably delete this message
and any copies.> -----Original Message-----
> From: ruslan usifov [mailto:ruslan.usifov@gmail.com]
> Sent: Wednesday, July 04, 2012 14:08
> To: user@cassandra.apache.org
> Subject: Tools for analize cassandra gc.log
>
> Hello
>
> Please dear community share some tools for gc.log analize. I found
> http://code.google.com/a/eclipselabs.org/p/garbagecat/, but it doesn't
> work

Mime
View raw message