cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hannu Kröger <hkro...@gmail.com>
Subject Re: Any way to control/limit off-heap memory?
Date Sun, 05 Mar 2017 12:34:38 GMT
If bloom filters are taking too much memory, you can adjust bloom filters:
https://docs.datastax.com/en/cassandra/2.1/cassandra/operations/ops_tuning_bloom_filters_c.html
<https://docs.datastax.com/en/cassandra/2.1/cassandra/operations/ops_tuning_bloom_filters_c.html>

Hannu

> On 4 Mar 2017, at 22:54, Thakrar, Jayesh <jthakrar@conversantmedia.com> wrote:
> 
> I have a situation where the off-heap memory is bloating the jvm process memory, making
it a candidate to be killed by the oom_killer.
> My server has 256 GB RAM and Cassandra heap memory of 16 GB
>  
> Below is the output of "nodetool info" and nodetool compactionstats for a culprit table
which causes bloom filter bloat.
> Ofcourse one option is to turnoff bloom filter, but I need to look into application access
pattern, etc.
>  
>  
> xss =  -ea -Dorg.xerial.snappy.tempdir=/home/vchadoop/var/tmp -javaagent:/home/vchadoop/apps/apache-cassandra-2.2.5//lib/jamm-0.3.0.jar
-XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -Xms16G -Xmx16G -Xmn4800M -XX:+HeapDumpOnOutOfMemoryError
-Xss256k
> ID                     : 2b9b4252-0760-49c1-8d14-544be0183271
> Gossip active          : true
> Thrift active          : false
> Native Transport active: true
> Load                   : 953.19 GB
> Generation No          : 1488641545
> Uptime (seconds)       : 15706
> Heap Memory (MB)       : 7692.93 / 16309.00
> Off Heap Memory (MB)   : 175115.07
> Data Center            : ord
> Rack                   : rack3
> Exceptions             : 0
> Key Cache              : entries 0, size 0 bytes, capacity 0 bytes, 0 hits, 0 requests,
NaN recent hit rate, 14400 save period in seconds
> Row Cache              : entries 0, size 0 bytes, capacity 0 bytes, 0 hits, 0 requests,
NaN recent hit rate, 0 save period in seconds
> Counter Cache          : entries 0, size 0 bytes, capacity 50 MB, 0 hits, 0 requests,
NaN recent hit rate, 7200 save period in seconds
> Token                  : (invoke with -T/--tokens to see all 256 tokens)
>  
>  
>                 Table: logs_by_user
>                 SSTable count: 622
>                 SSTables in each level: [174/4, 447/10, 0, 0, 0, 0, 0, 0, 0]
>                 Space used (live): 313156769247
>                 Space used (total): 313156769247
>                 Space used by snapshots (total): 0
>                 Off heap memory used (total): 180354511884
>                 SSTable Compression Ratio: 0.25016314078395613
>                 Number of keys (estimate): 147261312
>                 Memtable cell count: 44796
>                 Memtable data size: 57578717
>                 Memtable off heap memory used: 0
>                 Memtable switch count: 21
>                 Local read count: 0
>                 Local read latency: NaN ms
>                 Local write count: 1148687
>                 Local write latency: 0.123 ms
>                 Pending flushes: 0
>                 Bloom filter false positives: 0
>                 Bloom filter false ratio: 0.00000
>                 Bloom filter space used: 180269125192
>                 Bloom filter off heap memory used: 180269120216
>                 Index summary off heap memory used: 24335340
>                 Compression metadata off heap memory used: 61056328
>                 Compacted partition minimum bytes: 150
>                 Compacted partition maximum bytes: 668489532
>                 Compacted partition mean bytes: 3539
>                 Average live cells per slice (last five minutes): NaN
>                 Maximum live cells per slice (last five minutes): 0
>                 Average tombstones per slice (last five minutes): NaN
>                 Maximum tombstones per slice (last five minutes): 0
>  
>  
> From: Conversant <jthak <mailto:jthakrar@conversantmedia.com>

Mime
View raw message