cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benedict (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-7819) In progress compactions should not prevent deletion of stale sstables
Date Fri, 22 Aug 2014 14:06:12 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-7819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14106853#comment-14106853
] 

Benedict commented on CASSANDRA-7819:
-------------------------------------

Hmm. Yes, it looks like this _isn't_ safe with parallel compaction. However we can make it
safe by simply making the variable volatile.

> In progress compactions should not prevent deletion of stale sstables
> ---------------------------------------------------------------------
>
>                 Key: CASSANDRA-7819
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-7819
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Benedict
>            Assignee: Benedict
>            Priority: Minor
>              Labels: compaction
>             Fix For: 2.0.10
>
>         Attachments: 7819.txt
>
>
> Compactions retain references to potentially many sstables that existed when they were
started but that are now obsolete; many concurrent compactions can compound this dramatically,
and with very large files in size tiered compaction it is possible to inflate disk utilisation
dramatically beyond what is necessary.
> I propose, during compaction, periodically checking which sstables are obsolete and simply
replacing them with the sstable that replaced it. These sstables are by definition only used
for lookup, since we are in the process of obsoleting the sstables we're compacting, they're
only used to reference overlapping ranges which may be covered by tombstones.
> A simplest solution might even be to simply detect obsoletion and recalculate our overlapping
tree afresh. This is a pretty quick operation in the grand scheme of things, certainly wrt
compaction, so nothing lost to do this at the rate we obsolete sstables.
> See CASSANDRA-7139 for original discussion of the problem.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message