Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Date: Thu, 3 Sep 2015 20:22:46 +0000 (UTC)
From: "mike (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: <JIRA.12757014.1416588651000.250739.1441311766472@Atlassian.JIRA>
In-Reply-To: <JIRA.12757014.1416588651000@Atlassian.JIRA>
References: <JIRA.12757014.1416588651000@Atlassian.JIRA>
 <JIRA.12757014.1416588651729@arcas>
Subject: [jira] [Commented] (CASSANDRA-8359) Make DTCS consider removing
 SSTables much more frequently
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


    [ https://issues.apache.org/jira/browse/CASSANDRA-8359?page=3Dcom.atlas=
sian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D=
14729710#comment-14729710 ]=20

mike commented on CASSANDRA-8359:
---------------------------------

is there any way I can get a patch for 2.0.14?
Thanks!

> Make DTCS consider removing SSTables much more frequently
> ---------------------------------------------------------
>
>                 Key: CASSANDRA-8359
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-8359
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Bj=C3=B6rn Hegerfors
>            Assignee: Bj=C3=B6rn Hegerfors
>            Priority: Minor
>              Labels: dtcs
>             Fix For: 2.2.0 beta 1, 2.0.15, 2.1.5
>
>         Attachments: cassandra-2.0-CASSANDRA-8359.txt
>
>
> When I run DTCS on a table where every value has a TTL (always the same T=
TL), SSTables are completely expired, but still stay on disk for much longe=
r than they need to. I've applied CASSANDRA-8243, but it doesn't make an ap=
parent difference (probably because the subject SSTables are purged via com=
paction anyway, if not by directly dropping them).
> Disk size graphs show clearly that tombstones are only removed when the o=
ldest SSTable participates in compaction. In the long run, size on disk con=
tinually grows bigger. This should not have to happen. It should easily be =
able to stay constant, thanks to DTCS separating the expired data from the =
rest.
> I think checks for whether SSTables can be dropped should happen much mor=
e frequently. This is something that probably only needs to be tweaked for =
DTCS, but perhaps there's a more general place to put this. Anyway, my thin=
king is that DTCS should, on every call to getNextBackgroundTask, check whi=
ch SSTables can be dropped. It would be something like a call to Compaction=
Controller.getFullyExpiredSSTables with all non-compactingSSTables sent in =
as "compacting" and all other SSTables sent in as "overlapping". The return=
ed SSTables, if any, are then added to whichever set of SSTables that DTCS =
decides to compact. Then before the compaction happens, Cassandra is going =
to make another call to CompactionController.getFullyExpiredSSTables, where=
 it will see that it can just drop them.
> This approach has a bit of redundancy in that it needs to call Compaction=
Controller.getFullyExpiredSSTables twice. To avoid that, the code path for =
deciding SSTables to drop would have to be changed.
> (Side tracking a little here: I'm also thinking that tombstone compaction=
s could be considered more often in DTCS. Maybe even some kind of multi-SST=
able tombstone compaction involving the oldest couple of SSTables...)


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)