Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Date: Thu, 27 Feb 2014 19:27:20 +0000 (UTC)
From: "Marcus Eriksson (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: <JIRA.12693358.1391611324492.119734.1393529240641@arcas>
In-Reply-To: <JIRA.12693358.1391611324492@arcas>
References: <JIRA.12693358.1391611324492@arcas>
Subject: [jira] [Comment Edited] (CASSANDRA-6654) Droppable tombstones are
 not being removed from LCS table despite being above 20%
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/CASSANDRA-6654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13914587#comment-13914587 ] 

Marcus Eriksson edited comment on CASSANDRA-6654 at 2/27/14 7:25 PM:
---------------------------------------------------------------------

This is the way Cassandra checks if it can drop tombstones during compaction;
# Find all sstables that are not in the currently-being-compacted-set
# Find the maxDeletionTime for the row that is currently being compacted (it is basically writetime + ttl and since in your case you mix TTLs on a row, it is most likely writetime + 6 months)
# Check if the row exists in any of the other sstables where any data is older than maxDeletionTime, since that could mean that there are tombstones (expried data becomes a tombstone) that cover data in that sstable.
# If the row is present in one of those sstables, we cannot drop the tombstones in the row we are currently compacting.

So, I would say this is expected since you are bound to keep rows around for atleast 6 months due to mixing TTLs

Note that the behavior has improved in 2.1 (which is currently in beta), there we find a max purgeable timestamp and can drop tombstones older than that.


was (Author: krummas):
This is the way Cassandra checks if it can drop tombstones during compaction;
# Find all sstables that are not in the currently-being-compacted-set
# Find the maxDeletionTime for the row that is currently being compacted (it is basically writetime + ttl and since in your case you mix TTLs on a row, it is most likely writetime + 6 months)
# Check if the row exists in any of the other sstables where the oldest data is newer than maxDeletionTime, since that could mean that there are tombstones (expried data becomes a tombstone) that cover data in that sstable.
# If the row is present in one of those sstables, we cannot drop the tombstones in the row we are currently compacting.

So, I would say this is expected since you are bound to keep rows around for atleast 6 months due to mixing TTLs

Note that the behavior has improved in 2.1 (which is currently in beta), there we find a max purgeable timestamp and can drop tombstones older than that.


> Droppable tombstones are not being removed from LCS table despite being above 20%
> ---------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-6654
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-6654
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>         Environment: 1.2.13 VNodes with murmur3
>            Reporter: Keith Wright
>            Assignee: Marcus Eriksson
>         Attachments: Screen Shot 2014-02-05 at 9.38.20 AM.png, dtrlog.txt, repro3.py
>
>
> JMX is showing that one of our CQL3 LCS tables has a droppable tombstone ratio above 20% and increasing (currently at 28%).  Compactions are not falling behind and we are using the OOTB setting for this feature so I would expect not to go above 20% (will attach screen shot from JMX).   Table description:
> CREATE TABLE global_user (
>   user_id timeuuid,
>   app_id int,
>   type text,
>   name text,
>   extra_param map<text, text>,
>   last timestamp,
>   paid boolean,
>   sku_time map<text, timestamp>,
>   values map<timestamp, float>,
>   PRIMARY KEY (user_id, app_id, type, name)
> ) WITH
>   bloom_filter_fp_chance=0.100000 AND
>   caching='KEYS_ONLY' AND
>   comment='' AND
>   dclocal_read_repair_chance=0.000000 AND
>   gc_grace_seconds=86400 AND
>   read_repair_chance=0.100000 AND
>   replicate_on_write='true' AND
>   populate_io_cache_on_flush='false' AND
>   compaction={'sstable_size_in_mb': '160', 'class': 'LeveledCompactionStrategy'} AND
>   compression={'chunk_length_kb': '8', 'crc_check_chance': '0.1', 'sstable_compression': 'LZ4Compressor'}; 


--
This message was sent by Atlassian JIRA
(v6.1.5#6160)