cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Wee <peich...@gmail.com>
Subject Re: Leveled Compaction Strategy with a really intensive delete workload
Date Tue, 26 May 2015 04:17:22 GMT
Hi Stefano,

I did a quick test, it looks almost instant if you do alter but remember,
in my test machine, there are no loaded data yet and switching from stcs to
lcs.

cqlsh:jw_schema1> CREATE TABLE DogTypes ( block_id uuid, species text,
alias text, population varint, PRIMARY KEY (block_id) ) WITH caching =
'keys_only' and COMPACTION = {'class': 'SizeTieredCompactionStrategy'};
cqlsh:jw_schema1> desc table dogtypes;

CREATE TABLE jw_schema1.dogtypes (
    block_id uuid PRIMARY KEY,
    alias text,
    population varint,
    species text
) WITH bloom_filter_fp_chance = 0.01
    AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
    AND comment = ''
    AND compaction = {'min_threshold': '4', 'class':
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32'}
    AND compression = {'sstable_compression':
'org.apache.cassandra.io.compress.LZ4Compressor'}
    AND dclocal_read_repair_chance = 0.1
    AND default_time_to_live = 0
    AND gc_grace_seconds = 864000
    AND max_index_interval = 2048
    AND memtable_flush_period_in_ms = 0
    AND min_index_interval = 128
    AND read_repair_chance = 0.0
    AND speculative_retry = '99.0PERCENTILE';

cqlsh:jw_schema1> ALTER TABLE dogtypes WITH COMPACTION = {'class':
'LeveledCompactionStrategy', 'tombstone_threshold': '0.3',
'unchecked_tombstone_compaction': 'true'} ;
cqlsh:jw_schema1>




---------------------------------


INFO  [MigrationStage:1] 2015-05-26 12:12:25,867 ColumnFamilyStore.java:882
- Enqueuing flush of schema_keyspaces: 436 (0%) on-heap, 0 (0%) off-heap
INFO  [MemtableFlushWriter:146] 2015-05-26 12:12:25,869 Memtable.java:339 -
Writing Memtable-schema_keyspaces@6829883(138 serialized bytes, 3 ops,
0%/0% of on/off-heap limit)
INFO  [MemtableFlushWriter:146] 2015-05-26 12:12:26,173 Memtable.java:378 -
Completed flushing
/var/lib/cassandra/data/system/schema_keyspaces-b0f2235744583cdb9631c43e59ce3676/system-schema_keyspaces-ka-8-Data.db
(163 bytes) for commitlog position ReplayPosition(segmentId=1432265013436,
position=423408)
INFO  [CompactionExecutor:191] 2015-05-26 12:12:26,174
CompactionTask.java:140 - Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/schema_keyspaces-b0f2235744583cdb9631c43e59ce3676/system-schema_keyspaces-ka-6-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/schema_keyspaces-b0f2235744583cdb9631c43e59ce3676/system-schema_keyspaces-ka-5-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/schema_keyspaces-b0f2235744583cdb9631c43e59ce3676/system-schema_keyspaces-ka-8-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/schema_keyspaces-b0f2235744583cdb9631c43e59ce3676/system-schema_keyspaces-ka-7-Data.db')]
INFO  [MigrationStage:1] 2015-05-26 12:12:26,176 ColumnFamilyStore.java:882
- Enqueuing flush of schema_columnfamilies: 5307 (0%) on-heap, 0 (0%)
off-heap
INFO  [MemtableFlushWriter:147] 2015-05-26 12:12:26,178 Memtable.java:339 -
Writing Memtable-schema_columnfamilies@32790230(1380 serialized bytes, 27
ops, 0%/0% of on/off-heap limit)
INFO  [MemtableFlushWriter:147] 2015-05-26 12:12:26,550 Memtable.java:378 -
Completed flushing
/var/lib/cassandra/data/system/schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697/system-schema_columnfamilies-ka-7-Data.db
(922 bytes) for commitlog position ReplayPosition(segmentId=1432265013436,
position=423408)
INFO  [MigrationStage:1] 2015-05-26 12:12:26,598 ColumnFamilyStore.java:882
- Enqueuing flush of dogtypes: 0 (0%) on-heap, 0 (0%) off-heap
INFO  [CompactionExecutor:191] 2015-05-26 12:12:26,668
CompactionTask.java:270 - Compacted 4 sstables to
[/var/lib/cassandra/data/system/schema_keyspaces-b0f2235744583cdb9631c43e59ce3676/system-schema_keyspaces-ka-9,].
 751 bytes to 262 (~34% of original) in 492ms = 0.000508MB/s.  6 total
partitions merged to 3.  Partition merge counts were {1:2, 4:1, }


hth

jason

On Tue, May 26, 2015 at 6:24 AM, Stefano Ortolani <ostefano@gmail.com>
wrote:

> Ok, I am reading a bit more about compaction subproperties here (
> http://docs.datastax.com/en/cql/3.1/cql/cql_reference/compactSubprop.html)
> and it seems that tombstone_threshold and unchecked_tombstone_compaction
> might come handy.
>
> Does anybody know if changing any of these values (via ALTER) is possible
> without downtime, and how fast those values are picked up?
>
> Cheers,
> Stefano
>
>
> On Mon, May 25, 2015 at 1:32 PM, Stefano Ortolani <ostefano@gmail.com>
> wrote:
>
>> Hi all,
>>
>> Thanks for your answers! Yes, I agree that a delete intensive workload is
>> not something Cassandra is designed for.
>>
>> Unfortunately this is to cope with some unexpected data transformations
>> that I hope are a temporary thing.
>>
>> We chose LCS strategy because of really wide rows which were spanning
>> several SStables with other compaction strategies (and hence leading to
>> high latency read queries).
>>
>> I was honestly thinking of scraping and rebuilding the SStable from
>> scratch if this workload is confirmed to be temporary. Knowing the answer
>> to my question above would help second guess my a decision a bit less :)
>>
>> Cheers,
>> Stefano
>>
>> On Mon, May 25, 2015 at 9:52 AM, Jason Wee <peichieh@gmail.com> wrote:
>>
>>> ...., due to a really intensive delete workloads, the SSTable is
>>> promoted to t......
>>>
>>> Is cassandra design for *delete* workloads? doubt so. Perhaps looking at
>>> some other alternative like ttl?
>>>
>>> jason
>>>
>>> On Mon, May 25, 2015 at 10:12 AM, Manoj Khangaonkar <
>>> khangaonkar@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> For a delete intensive workload ( translate to write intensive), is
>>>> there any reason to use leveled compaction ? The recommendation seems to
be
>>>> that leveled compaction is suited for read intensive workloads.
>>>>
>>>> Depending on your use case, you might better of with data tiered or
>>>> size tiered strategy.
>>>>
>>>> regards
>>>>
>>>> regards
>>>>
>>>> On Sun, May 24, 2015 at 10:50 AM, Stefano Ortolani <ostefano@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> I have a question re leveled compaction strategy that has been bugging
>>>>> me quite a lot lately. Based on what I understood, a compaction takes
place
>>>>> when the SSTable gets to a specific size (10 times the size of its previous
>>>>> generation). My question is about an edge case where, due to a really
>>>>> intensive delete workloads, the SSTable is promoted to the next level
(say
>>>>> L1) and its size, because of the many evicted tombstones, fall back to
1/10
>>>>> of its size (hence to a size compatible to the previous generation, L0).
>>>>>
>>>>> What happens in this case? If the next major compaction is set to
>>>>> happen when the SSTable is promoted to L2, well, that might take too
long
>>>>> and too many tobmstones could then appear in the meanwhile (and queries
>>>>> might subsequently fail). Wouldn't be more correct to flag the SStable's
>>>>> generation to its previous value (namely, not changing it even if a major
>>>>> compaction took place)?
>>>>>
>>>>> Regards,
>>>>> Stefano Ortolani
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> http://khangaonkar.blogspot.com/
>>>>
>>>
>>>
>>
>

Mime
View raw message