cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keith Wright <>
Subject Re: need help with Cassandra 1.2 Full GCing -- output of jmap histogram
Date Mon, 10 Mar 2014 13:46:28 GMT
I also want to point out an issue I filed that was closed as not an issue:  CASSANDRA-6654
  Basically if you’re using mixed TTLs on columns in a row, the “oldest” TTL is used
to determine if tombstones of other columns can be removed. In other words, if you have a
column with a 1 day TTL and a column with a 1 year TTL, the 1 day TTLed data (now tombstoned)
will not be removed on compaction.

From: Jonathan Lacefield <<>>
Reply-To: "<>" <<>>
Date: Monday, March 10, 2014 at 8:33 AM
To: "<>" <<>>
Subject: Re: need help with Cassandra 1.2 Full GCing -- output of jmap histogram


  You have several options:

  1) going forward lower gc_grace_seconds
       - this is very use case specific.  Default is 10 days.  Some users will put this at
0 for specific use cases.
  2) you could also lower tombstone compaction threshold and interval to get tombstone compaction
to fire more often on your tables/cfs:
  3) to clean out old tombstones you could always run a manual compaction, those these aren't
typically recommended though:

  For 1 and 2, be sure your disks can keep up with compaction to ensure tombstone, or other,
compaction fires regularly enough to clean out old tombstones.  Also, you probably want to
ensure you are using Level Compaction:
 Again, this assumes your disk system can handle the increased io from Leveled Compaction.

  Also, you may be running into this with the older version of Cassandra:

  Hope this helps.


Jonathan Lacefield
Solutions Architect, DataStax
(404) 822 3487


On Mon, Mar 10, 2014 at 6:41 AM, Oleg Dulin <<>>
I get that :)

What I'd like to know is how to fix that :)

On 2014-03-09 20:24:54 +0000, Takenori Sato said:

You have millions of org.apache.cassandra.db.DeletedColumn instances on the snapshot.

This means you have lots of column tombstones, and I guess, which are read into memory by
slice query.

On Sun, Mar 9, 2014 at 10:55 PM, Oleg Dulin <<>>
I am trying to understand why one of my nodes keeps full GC.

I have Xmx set to 8gigs, memtable total size is 2 gigs.

Consider the top entries from jmap -histo:live @

Oleg Dulin

Oleg Dulin

View raw message