cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marcus Eriksson (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-6654) Droppable tombstones are not being removed from LCS table despite being above 20%
Date Thu, 27 Feb 2014 19:27:20 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-6654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13914587#comment-13914587
] 

Marcus Eriksson edited comment on CASSANDRA-6654 at 2/27/14 7:25 PM:
---------------------------------------------------------------------

This is the way Cassandra checks if it can drop tombstones during compaction;
# Find all sstables that are not in the currently-being-compacted-set
# Find the maxDeletionTime for the row that is currently being compacted (it is basically
writetime + ttl and since in your case you mix TTLs on a row, it is most likely writetime
+ 6 months)
# Check if the row exists in any of the other sstables where any data is older than maxDeletionTime,
since that could mean that there are tombstones (expried data becomes a tombstone) that cover
data in that sstable.
# If the row is present in one of those sstables, we cannot drop the tombstones in the row
we are currently compacting.

So, I would say this is expected since you are bound to keep rows around for atleast 6 months
due to mixing TTLs

Note that the behavior has improved in 2.1 (which is currently in beta), there we find a max
purgeable timestamp and can drop tombstones older than that.



was (Author: krummas):
This is the way Cassandra checks if it can drop tombstones during compaction;
# Find all sstables that are not in the currently-being-compacted-set
# Find the maxDeletionTime for the row that is currently being compacted (it is basically
writetime + ttl and since in your case you mix TTLs on a row, it is most likely writetime
+ 6 months)
# Check if the row exists in any of the other sstables where the oldest data is newer than
maxDeletionTime, since that could mean that there are tombstones (expried data becomes a tombstone)
that cover data in that sstable.
# If the row is present in one of those sstables, we cannot drop the tombstones in the row
we are currently compacting.

So, I would say this is expected since you are bound to keep rows around for atleast 6 months
due to mixing TTLs

Note that the behavior has improved in 2.1 (which is currently in beta), there we find a max
purgeable timestamp and can drop tombstones older than that.


> Droppable tombstones are not being removed from LCS table despite being above 20%
> ---------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-6654
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-6654
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>         Environment: 1.2.13 VNodes with murmur3
>            Reporter: Keith Wright
>            Assignee: Marcus Eriksson
>         Attachments: Screen Shot 2014-02-05 at 9.38.20 AM.png, dtrlog.txt, repro3.py
>
>
> JMX is showing that one of our CQL3 LCS tables has a droppable tombstone ratio above
20% and increasing (currently at 28%).  Compactions are not falling behind and we are using
the OOTB setting for this feature so I would expect not to go above 20% (will attach screen
shot from JMX).   Table description:
> CREATE TABLE global_user (
>   user_id timeuuid,
>   app_id int,
>   type text,
>   name text,
>   extra_param map<text, text>,
>   last timestamp,
>   paid boolean,
>   sku_time map<text, timestamp>,
>   values map<timestamp, float>,
>   PRIMARY KEY (user_id, app_id, type, name)
> ) WITH
>   bloom_filter_fp_chance=0.100000 AND
>   caching='KEYS_ONLY' AND
>   comment='' AND
>   dclocal_read_repair_chance=0.000000 AND
>   gc_grace_seconds=86400 AND
>   read_repair_chance=0.100000 AND
>   replicate_on_write='true' AND
>   populate_io_cache_on_flush='false' AND
>   compaction={'sstable_size_in_mb': '160', 'class': 'LeveledCompactionStrategy'} AND
>   compression={'chunk_length_kb': '8', 'crc_check_chance': '0.1', 'sstable_compression':
'LZ4Compressor'}; 



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message