cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From shalom sagges <shalomsag...@gmail.com>
Subject Re: Large Partitions
Date Mon, 02 Apr 2018 22:48:52 GMT
Thanks Ali!

I use a 13 months TTL on this table. I guess I need to remodel this table.
And I'll definitely try this tool.



On Tue, Apr 3, 2018 at 1:28 AM, Ali Hubail <Ali.Hubail@petrolink.com> wrote:

> system.log should show you some warnings about wide rows. Do a grep on
> system.log for 'Writing large partition' The message could be different for
> the c* version you're using though. Plus, this doesn't show you all of the
> large partitions.
>
> There is a nice tool that analyzes sstables and can show the large
> partitions:
> https://github.com/tolbertam/sstable-tools
>
>
> By "how to deal with them?" it depends. If you don't need those
> partitions then you can delete them. You can also use TTL if it fits you or
> remodel your table to only hold upto 100k rows or 100mb per partition
> (whichever comes first). If you're going to remodel the table, aim for much
> less than 100k/100mb per partition.
>
> *Ali Hubail*
>
> Confidentiality warning: This message and any attachments are intended
> only for the persons to whom this message is addressed, are confidential,
> and may be privileged. If you are not the intended recipient, you are
> hereby notified that any review, retransmission, conversion to hard copy,
> copying, modification, circulation or other use of this message and any
> attachments is strictly prohibited. If you receive this message in error,
> please notify the sender immediately by return email, and delete this
> message and any attachments from your system. Petrolink International
> Limited its subsidiaries, holding companies and affiliates disclaims all
> responsibility from and accepts no liability whatsoever for the
> consequences of any unauthorized person acting, or refraining from acting,
> on any information contained in this message. For security purposes, staff
> training, to assist in resolving complaints and to improve our customer
> service, email communications may be monitored and telephone calls may be
> recorded.
>
>
> *shalom sagges <shalomsagges@gmail.com <shalomsagges@gmail.com>>*
>
> 04/02/2018 03:57 PM
> Please respond to
> user@cassandra.apache.org
>
> To
> user@cassandra.apache.org,
>
> cc
> Subject
> Large Partitions
>
>
>
>
> Hi All,
>
> I ran nodetool cfstats (v2.0.14) on a keyspace and found that there are a
> few large partitions. I assume that since "Compacted partition maximum
> bytes": 802187438 (~800 MB) and since
> "Compacted partition mean bytes": 100465 (~100 KB), it means that most
> partitions are in okay size and only a few are large. Am I assuming
> correctly?
>
> If so, can anyone suggest how to find those large partitions and how to
> deal with them? (cfstats output below)
>
> Thanks!
>
>
> nodetool cfstats keyspace1;
>
> Table: table1
> SSTable count: 16
> Space used (live), bytes: 453844035587
> Space used (total), bytes: 453844035587
> Off heap memory used (total), bytes: 440787635
> SSTable Compression Ratio: 0.17417149031966575
> Number of keys (estimate): 33651200
> Memtable cell count: 27966
> Memtable data size, bytes: 41698140
> Memtable switch count: 199727
> Local read count: 86494530
> Local read latency: 2.646 ms
> Local write count: 247712138
> Local write latency: 0.030 ms
> Pending tasks: 0
> Bloom filter false positives: 2182242
> Bloom filter false ratio: 0.02251
> Bloom filter space used, bytes: 53135136
> Bloom filter off heap memory used, bytes: 53135008
> Index summary off heap memory used, bytes: 11560419
> Compression metadata off heap memory used, bytes: 376092208
> Compacted partition minimum bytes: 373
>
> * Compacted partition maximum bytes: 802187438 Compacted partition mean
> bytes: 100465*
> Average live cells per slice (last five minutes): 37.0
> Average tombstones per slice (last five minutes): 0.0
>
>

Mime
View raw message