cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From shalom sagges <shalomsag...@gmail.com>
Subject Re: User Defined Compaction Issue
Date Tue, 26 Sep 2017 12:53:36 GMT
Awesome explanation :-)

Thanks a lot!

On Tue, Sep 26, 2017 at 3:40 PM, Jeff Jirsa <jjirsa@gmail.com> wrote:

> Write row A, flush into sstable 1
> Delete row A, flush the tombstone into sstable 2
>
> The tombstone in sstable 2 can’t be removed until row A in sstable 1 gets
> removed. If you just keep recompacting sstable 2 by itself, the row in
> sstable A remains on disk.
>
>
>
> --
> Jeff Jirsa
>
>
> On Sep 26, 2017, at 2:01 AM, shalom sagges <shalomsagges@gmail.com> wrote:
>
> Thanks Jeff!
>
> I'll try that.
> I'm not sure I understand how the tombstones are covering data in another
> file. Do you have a small example perhaps?
>
> Thanks again!
>
> On Tue, Sep 26, 2017 at 1:38 AM, Jeff Jirsa <jjirsa@gmail.com> wrote:
>
>> The problem is likely that your sstables overlap - your 91% droppable
>> tombstones are probably covering data in another file. Until that data is
>> removed, those tombstones cant be dropped.
>>
>> This is why a full major compaction often works better for simply
>> reclaiming disk space (though it takes a lot more disk as it runs and has
>> other drawbacks) - it joins the shadowed data with the tombstones into a
>> single file. You can sorta fake that by using user-defined compaction with
>> multiple sstables to try to compact away the shadowed data - pick 2 or more
>> -Data files and compact them together. Works best if you can be sure which
>> data is overlapping, but short of that you'll probably want to pick data
>> with approximately the same (or older) calendar timestamps.
>>
>>
>>
>> On Mon, Sep 25, 2017 at 11:10 AM, shalom sagges <shalomsagges@gmail.com>
>> wrote:
>>
>>> Hi Everyone,
>>>
>>> I'm running into an issue I can't seem to Solve.
>>>
>>> I execute force compaction in order to reclaim back storage.
>>> Everything was working fine for a time, but after a while I found that
>>> tombstones aren't being removed any longer.
>>>
>>> For example, I've compacted the following SSTable:
>>> *21G *Sep 19 10:18 file-jb-69103-Data.db
>>> Estimated droppable tombstones:
>>>
>>> *0.9115601492568154*
>>> I ran it with jmxterm and the compaction started and completed
>>> successfully.
>>>  $>run -b org.apache.cassandra.db:type=CompactionManager
>>> forceUserDefinedCompaction file-jb-69103-Data.db
>>>  #calling operation forceUserDefinedCompaction of mbean
>>> org.apache.cassandra.db:type=CompactionManager
>>>  #operation returns:
>>> null
>>>
>>>
>>> A new SStable was created, but no tombstones were removed.
>>>  *21G *Sep 25 12:16 file-jb-71319-Data.db
>>>  Estimated droppable tombstones:
>>>
>>> * 0.9115601492568154*
>>> I've tried running it from jconsole as well, but the outcome was the
>>> same.
>>>
>>> Is anyone familiar with this issue?
>>> Btw, this cluster is on version 2.0.14 .
>>>
>>>
>>> Thanks!
>>> Shalom
>>>
>>>
>>
>

Mime
View raw message