cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ali Hubail <Ali.Hub...@petrolink.com>
Subject Re: Large Partitions
Date Mon, 02 Apr 2018 22:28:48 GMT
system.log should show you some warnings about wide rows. Do a grep on 
system.log for 'Writing large partition' The message could be different 
for the c* version you're using though. Plus, this doesn't show you all of 
the large partitions.

There is a nice tool that analyzes sstables and can show the large 
partitions:
https://github.com/tolbertam/sstable-tools


By "how to deal with them?" it depends. If you don't need those partitions 
then you can delete them. You can also use TTL if it fits you or remodel 
your table to only hold upto 100k rows or 100mb per partition (whichever 
comes first). If you're going to remodel the table, aim for much less than 
100k/100mb per partition.

Ali Hubail

Confidentiality warning: This message and any attachments are intended 
only for the persons to whom this message is addressed, are confidential, 
and may be privileged. If you are not the intended recipient, you are 
hereby notified that any review, retransmission, conversion to hard copy, 
copying, modification, circulation or other use of this message and any 
attachments is strictly prohibited. If you receive this message in error, 
please notify the sender immediately by return email, and delete this 
message and any attachments from your system. Petrolink International 
Limited its subsidiaries, holding companies and affiliates disclaims all 
responsibility from and accepts no liability whatsoever for the 
consequences of any unauthorized person acting, or refraining from acting, 
on any information contained in this message. For security purposes, staff 
training, to assist in resolving complaints and to improve our customer 
service, email communications may be monitored and telephone calls may be 
recorded.



shalom sagges <shalomsagges@gmail.com> 
04/02/2018 03:57 PM
Please respond to
user@cassandra.apache.org


To
user@cassandra.apache.org, 
cc

Subject
Large Partitions






Hi All, 

I ran nodetool cfstats (v2.0.14) on a keyspace and found that there are a 
few large partitions. I assume that since "Compacted partition maximum 
bytes": 802187438 (~800 MB) and since 
"Compacted partition mean bytes": 100465 (~100 KB), it means that most 
partitions are in okay size and only a few are large. Am I assuming 
correctly?

If so, can anyone suggest how to find those large partitions and how to 
deal with them? (cfstats output below)

Thanks! 


nodetool cfstats keyspace1;

Table: table1
SSTable count: 16
Space used (live), bytes: 453844035587
Space used (total), bytes: 453844035587
Off heap memory used (total), bytes: 440787635
SSTable Compression Ratio: 0.17417149031966575
Number of keys (estimate): 33651200
Memtable cell count: 27966
Memtable data size, bytes: 41698140
Memtable switch count: 199727
Local read count: 86494530
Local read latency: 2.646 ms
Local write count: 247712138
Local write latency: 0.030 ms
Pending tasks: 0
Bloom filter false positives: 2182242
Bloom filter false ratio: 0.02251
Bloom filter space used, bytes: 53135136
Bloom filter off heap memory used, bytes: 53135008
Index summary off heap memory used, bytes: 11560419
Compression metadata off heap memory used, bytes: 376092208
Compacted partition minimum bytes: 373
Compacted partition maximum bytes: 802187438
Compacted partition mean bytes: 100465
Average live cells per slice (last five minutes): 37.0
Average tombstones per slice (last five minutes): 0.0


Mime
View raw message