cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Dulin <oleg.du...@gmail.com>
Subject Re: need help with Cassandra 1.2 Full GCing -- output of jmap histogram
Date Tue, 11 Mar 2014 19:24:50 GMT
Good news is that since I lowered gc_grace period it collected over 
100Gigs of tombstones and seems much happier now.

Oleg

On 2014-03-10 13:33:43 +0000, Jonathan Lacefield said:

> Hello,
> 
>   You have several options:
> 
>   1) going forward lower gc_grace_seconds 
> http://www.datastax.com/documentation/cassandra/1.2/cassandra/configuration/configStorage_r.html?pagename=docs&version=1.2&file=configuration/storage_configuration#gc-grace-seconds

> 
>        - this is very use case specific.  Default is 10 days.  Some 
> users will put this at 0 for specific use cases.
>   2) you could also lower tombstone compaction threshold and interval 
> to get tombstone compaction to fire more often on your tables/cfs:  
> https://datastax.jira.com/wiki/pages/viewpage.action?pageId=54493436
>   3) to clean out old tombstones you could always run a manual 
> compaction, those these aren't typically recommended though:  
> http://www.datastax.com/documentation/cassandra/1.2/cassandra/tools/toolsNodetool_r.html

> 
>        
>   For 1 and 2, be sure your disks can keep up with compaction to ensure 
> tombstone, or other, compaction fires regularly enough to clean out old 
> tombstones.  Also, you probably want to ensure you are using Level 
> Compaction:  
> http://www.datastax.com/dev/blog/when-to-use-leveled-compaction.  
> Again, this assumes your disk system can handle the increased io from 
> Leveled Compaction.
> 
>   Also, you may be running into this with the older version of 
> Cassandra: https://issues.apache.org/jira/browse/CASSANDRA-6541
> 
>   Hope this helps.
> 
> Jonathan
> 
> 
> Jonathan Lacefield
> Solutions Architect, DataStax
> (404) 822 3487
> <image>
> 
> 
> <image>
> 
> 
> On Mon, Mar 10, 2014 at 6:41 AM, Oleg Dulin <oleg.dulin@gmail.com> wrote:
> I get that :)
> 
> What I'd like to know is how to fix that :)
> 
> On 2014-03-09 20:24:54 +0000, Takenori Sato said:
> 
> You have millions of org.apache.cassandra.db.DeletedColumn instances on 
> the snapshot.
> 
> This means you have lots of column tombstones, and I guess, which are 
> read into memory by slice query. 
> 
> 
> On Sun, Mar 9, 2014 at 10:55 PM, Oleg Dulin <oleg.dulin@gmail.com> wrote:
> I am trying to understand why one of my nodes keeps full GC.
> 
> I have Xmx set to 8gigs, memtable total size is 2 gigs.
> 
> Consider the top entries from jmap -histo:live @ http://pastebin.com/UaatHfpJ
> 
> --
> Regards,
> Oleg Dulin
> http://www.olegdulin.com
> 
> 
> --
> Regards,
> Oleg Dulin
> http://www.olegdulin.com

S
-- 
Regards,
Oleg Dulin
http://www.olegdulin.com



Mime
View raw message