cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Venkatesh Arivazhagan <venkey.a...@gmail.com>
Subject Re: Help with tombstones and compaction
Date Tue, 22 Sep 2015 21:03:39 GMT
Thank you so much Jeff! I will bump to Cassandra 2.1.9 and update this
thread with the results!

Thanks,
Venkatesh

On Mon, Sep 21, 2015 at 2:19 PM, Jeff Jirsa <jeff.jirsa@crowdstrike.com>
wrote:

> The timestamp involed here isn’t the one defined in the schema, it’s the
> timestamp written on each cell when you apply a mutation (write).
>
> That timestamp is the one returned by WRITETIME(), and visible in
> sstablemetadata – it’s not visible in the schema directly.
>
> Failing to have the proper unit (milliseconds vs microseconds) could
> certainly cause confusion.
>
>
>
> From: Venkatesh Arivazhagan
> Reply-To: "user@cassandra.apache.org"
> Date: Monday, September 21, 2015 at 1:41 PM
> To: "user@cassandra.apache.org"
> Subject: Re: Help with tombstones and compaction
>
> Thank you for your reply Jeff!
>
> I will switch to Cassandra 2.1.9.
>
> Quick follow up question: Does the schema, settings I have setup look
> alright? My timestamp column's type is blob - I was wondering if this could
> confuse DTCS?
>
> On Sun, Sep 20, 2015 at 3:37 PM, Jeff Jirsa <jeff.jirsa@crowdstrike.com>
> wrote:
>
>> 2.1.4 is getting pretty old. There’s a DTCS deletion tweak in 2.1.5 (
>> https://issues.apache.org/jira/browse/CASSANDRA-8359 ) that may help you.
>>
>> 2.1.5 and 2.1.6 have some memory leak issues in DTCS, so go to 2.1.7 or
>> newer (probably 2.1.9 unless you have a compelling reason not to go to
>> 2.1.9)
>>
>>
>> From: Venkatesh Arivazhagan
>> Reply-To: "user@cassandra.apache.org"
>> Date: Sunday, September 20, 2015 at 2:48 PM
>> To: "user@cassandra.apache.org"
>> Subject: Help with tombstones and compaction
>>
>> Hi Guys,
>>
>> I have a Cassandra 2.1.4 cluster with 14 nodes. I am using it primarily
>> for storing time series data collected via KairosDB.
>> The default TTL for data inserted into the column family named
>> data_points is 12hrs. I have also set the gc_grace_seconds to 12 hrs.
>> In-spite of this my disk space keeps on increasing and it looks like
>> tombstones are never dropped.
>>
>> It looks like compactions are happening on a regular basis. The SSTable
>> count does not seem outrageous either. It is constantly between ~10 to ~22.
>>
>> Am I doing anything wrong? Is there a way to mitigate this?
>>
>> Attached:
>> * DESC output for my keyspace
>> * Disk usage graph
>> * LiveSSTable Count graph
>>
>>
>> --------------------------------------------------------------------------------------------------------------------------
>>
>> CREATE KEYSPACE kairosdb WITH replication = {'class': 'SimpleStrategy',
>> 'replication_factor': '3'}  AND durable_writes = true;
>>
>> CREATE TABLE kairosdb.data_points (
>>     key blob,
>>     column1 blob,
>>     value blob,
>>     PRIMARY KEY (key, column1)
>> ) WITH COMPACT STORAGE
>>     AND CLUSTERING ORDER BY (column1 ASC)
>>     AND bloom_filter_fp_chance = 0.01
>>     AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
>>     AND comment = ''
>>     AND compaction = {'max_sstable_age_days': '365', 'base_time_seconds':
>> '3600', 'max_threshold': '32', 'timestamp_resolution': 'MILLISECONDS',
>> 'enabled': 'true', 'tombstone_compaction_interval': '1', 'min_threshold':
>> '4', 'tombstone_threshold': '.1', 'class':
>> 'org.apache.cassandra.db.compaction.DateTieredCompactionStrategy'}
>>     AND compression = {'sstable_compression':
>> 'org.apache.cassandra.io.compress.LZ4Compressor'}
>>     AND dclocal_read_repair_chance = 0.1
>>     AND default_time_to_live = 43200
>>     AND gc_grace_seconds = 43200
>>     AND max_index_interval = 2048
>>     AND memtable_flush_period_in_ms = 0
>>     AND min_index_interval = 128
>>     AND read_repair_chance = 0.1
>>     AND speculative_retry = 'NONE';
>>
>> CREATE TABLE kairosdb.row_key_index (
>>     key blob,
>>     column1 blob,
>>     value blob,
>>     PRIMARY KEY (key, column1)
>> ) WITH COMPACT STORAGE
>>     AND CLUSTERING ORDER BY (column1 ASC)
>>     AND bloom_filter_fp_chance = 0.01
>>     AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
>>     AND comment = ''
>>     AND compaction = {'min_threshold': '4', 'class':
>> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
>> 'max_threshold': '32'}
>>     AND compression = {'sstable_compression':
>> 'org.apache.cassandra.io.compress.LZ4Compressor'}
>>     AND dclocal_read_repair_chance = 0.1
>>     AND default_time_to_live = 0
>>     AND gc_grace_seconds = 43200
>>     AND max_index_interval = 2048
>>     AND memtable_flush_period_in_ms = 0
>>     AND min_index_interval = 128
>>     AND read_repair_chance = 0.1
>>     AND speculative_retry = 'NONE';
>>
>> CREATE TABLE kairosdb.string_index (
>>     key blob,
>>     column1 text,
>>     value blob,
>>     PRIMARY KEY (key, column1)
>> ) WITH COMPACT STORAGE
>>     AND CLUSTERING ORDER BY (column1 ASC)
>>     AND bloom_filter_fp_chance = 0.01
>>     AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
>>     AND comment = ''
>>     AND compaction = {'min_threshold': '4', 'class':
>> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
>> 'max_threshold': '32'}
>>     AND compression = {'sstable_compression':
>> 'org.apache.cassandra.io.compress.LZ4Compressor'}
>>     AND dclocal_read_repair_chance = 0.1
>>     AND default_time_to_live = 0
>>     AND gc_grace_seconds = 43200
>>     AND max_index_interval = 2048
>>     AND memtable_flush_period_in_ms = 0
>>     AND min_index_interval = 128
>>     AND read_repair_chance = 0.1
>>     AND speculative_retry = 'NONE';
>>
>> --------------------------------------------------------------------------------------------------------------------------
>>
>
>

Mime
View raw message