cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nicolas Guyomar <nicolas.guyo...@gmail.com>
Subject Re: old big tombstone data file occupy much disk space
Date Mon, 04 Sep 2017 11:54:25 GMT
You'll get the WARN "Will not compact {}: it is not an active sstable"  :)

On 4 September 2017 at 12:07, Shalom Sagges <shaloms@liveperson.com> wrote:

> By the way, does anyone know what happens if I run a user defined
> compaction on an sstable that's already in compaction?
>
>
>
>
>
>
> On Sun, Sep 3, 2017 at 2:55 PM, Shalom Sagges <shaloms@liveperson.com>
> wrote:
>
>> Try this blog by The Last Pickle:
>>
>> http://thelastpickle.com/blog/2016/10/18/user-defined-compaction.html
>>
>>
>>
>>
>>
>>
>> Shalom Sagges
>> DBA
>> <http://www.linkedin.com/company/164748> <http://twitter.com/liveperson>
>> <http://www.facebook.com/LivePersonInc> We Create Meaningful Connections
>>
>>
>>
>> On Sat, Sep 2, 2017 at 8:34 PM, Jeff Jirsa <jjirsa@gmail.com> wrote:
>>
>>> If you're on 3.0 (3.0.6 or 3.0.8 or newer I don't remember which), TWCS
>>> was designed for ttl-only time series use cases
>>>
>>> Alternatively, if you have IO to spare, you may find LCS works as well
>>> (it'll cause quite a bit more compaction, but a much higher chance to
>>> compact away tombstones)
>>>
>>> There are also tombstone focused sub properties to more aggressively
>>> compact sstables that have a lot of tombstones - check the docs for
>>> "unchecked tombstone compaction" and "tombstone threshold" - enabling those
>>> will enable more aggressive automatic single-sstable compactions
>>>
>>> --
>>> Jeff Jirsa
>>>
>>>
>>> On Sep 2, 2017, at 7:10 AM, qf zhou <zhouqf2013@gmail.com> wrote:
>>>
>>>
>>> Yes, your are right. I am using STCS compaction strategy with some kind
>>> of timeseries model. Too much disk space has been occupied.
>>>
>>> What should I  do to stop  the  disk full ?
>>>
>>>  I only want to keep 100 days data most recently,  so I set default_time_to_live
>>> = 8640000(100 days ).
>>>
>>> I know I need to do something to stop the disk space cost, but I really
>>> don’t know how to do it.
>>>
>>>
>>> Here is the strategy of the big data table :
>>>
>>>     AND compaction = {'class': 'org.apache.cassandra.db.compa
>>> ction.SizeTieredCompactionStrategy', 'max_threshold': '32',
>>> 'min_threshold': '12', 'tombstone_threshold': '0.1',
>>> 'unchecked_tombstone_compaction': 'true'}
>>>     AND compression = {'chunk_length_in_kb': '64', 'class': '
>>> org.apache.cassandra.io.compress.LZ4Compressor'}
>>>     AND crc_check_chance = 1.0
>>>     AND dclocal_read_repair_chance = 0.1
>>>     AND default_time_to_live = 8640000
>>>     AND gc_grace_seconds = 432000
>>>
>>>
>>>
>>> 在 2017年9月2日,下午7:34,Nicolas Guyomar <nicolas.guyomar@gmail.com>
写道:
>>>
>>> your are using STCS compaction strategy with some kind of timeseries
>>> model, and you are going to end up with yor disk full!
>>>
>>>
>>>
>>
>
> This message may contain confidential and/or privileged information.
> If you are not the addressee or authorized to receive this on behalf of
> the addressee you must not use, copy, disclose or take action based on this
> message or any information herein.
> If you have received this message in error, please advise the sender
> immediately by reply email and delete this message. Thank you.
>

Mime
View raw message