cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keith Wright <kwri...@nanigans.com>
Subject Re: Full GC in cassandra
Date Mon, 28 Jul 2014 16:07:47 GMT
What’s your cfhistograms look like?

From: Ruchir Jha <ruchir.jha@gmail.com<mailto:ruchir.jha@gmail.com>>
Reply-To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>>
Date: Monday, July 28, 2014 at 10:43 AM
To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>>
Subject: Re: Full GC in cassandra

Also we do subsequent updates (atleat 4) for each piece of data that we write.


On Mon, Jul 28, 2014 at 10:36 AM, Ruchir Jha <ruchir.jha@gmail.com<mailto:ruchir.jha@gmail.com>>
wrote:
Doing about 5K writes / second. Avg Data Size = 1.6 TB / node. Total Data Size = 21 TB.

And this is the nodetool cfstats output for one of our busiest column families:

  SSTable count: 10
                Space used (live): 43239294899
                Space used (total): 43239419603
                SSTable Compression Ratio: 0.2954468408497778
                Number of Keys (estimate): 63729152
                Memtable Columns Count: 1921620
                Memtable Data Size: 257680020
                Memtable Switch Count: 9
                Read Count: 6167
                Read Latency: NaN ms.
                Write Count: 770984
                Write Latency: 0.098 ms.
                Pending Tasks: 0
                Bloom Filter False Positives: 370
                Bloom Filter False Ratio: 0.00000
                Bloom Filter Space Used: 80103200
                Compacted row minimum size: 180
                Compacted row maximum size: 3311
                Compacted row mean size: 2631
                Average live cells per slice (last five minutes): 73.0
                Average tombstones per slice (last five minutes): 13.0



On Mon, Jul 28, 2014 at 10:14 AM, Mark Reddy <mark.reddy@boxever.com<mailto:mark.reddy@boxever.com>>
wrote:
What is your data size and number of columns in Cassandra. Do you do many deletions?


On Mon, Jul 28, 2014 at 2:53 PM, Ruchir Jha <ruchir.jha@gmail.com<mailto:ruchir.jha@gmail.com>>
wrote:
Really curious to know what's causing the spike in Columns and DeletedColums below :


2014-07-28T09:30:27.471-0400: 127335.928: [Full GC 127335.928: [Class Histogram:
 num     #instances         #bytes  class name
----------------------------------------------
   1:     132626060     6366050880<tel:6366050880>  java.nio.HeapByteBuffer
   2:      28194918     3920045528  [B
   3:      78124737     3749987376  edu.stanford.ppl.concurrent.SnapTreeMap$Node
   4:      67650128     2164804096<tel:2164804096>  org.apache.cassandra.db.Column
   5:      16315310      522089920  org.apache.cassandra.db.DeletedColumn
   6:          6818      392489608  [I
   7:       2844374      273059904  edu.stanford.ppl.concurrent.CopyOnWriteManager$COWEpoch
   8:       5727000      229080000  java.util.TreeMap$Entry
   9:        767742      182921376  [J
  10:       2932832      140775936  edu.stanford.ppl.concurrent.SnapTreeMap$RootHolder
  11:       2844375       91020000  edu.stanford.ppl.concurrent.CopyOnWriteManager$Latch
  12:       4145131       66322096  java.util.concurrent.atomic.AtomicReference
  13:        437874       64072392  [C
  14:       2660844       63860256  java.util.concurrent.ConcurrentSkipListMap$Node
  15:          4920       62849864  [[B
  16:       1632063       52226016  edu.stanford.ppl.concurrent.SnapTreeMap




Mime
View raw message