cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marcus Eriksson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-7819) In progress compactions should not prevent deletion of stale sstables
Date Thu, 02 Oct 2014 07:12:34 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-7819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156154#comment-14156154
] 

Marcus Eriksson commented on CASSANDRA-7819:
--------------------------------------------

bq. or simply delay this change until 2.1.

yeah lets do that, its a bit of a scary change with the ParallelCI still around

> In progress compactions should not prevent deletion of stale sstables
> ---------------------------------------------------------------------
>
>                 Key: CASSANDRA-7819
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-7819
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Benedict
>            Assignee: Benedict
>            Priority: Minor
>              Labels: compaction
>             Fix For: 2.0.11
>
>         Attachments: 0001-7819-v2.patch, 7819.txt
>
>
> Compactions retain references to potentially many sstables that existed when they were
started but that are now obsolete; many concurrent compactions can compound this dramatically,
and with very large files in size tiered compaction it is possible to inflate disk utilisation
dramatically beyond what is necessary.
> I propose, during compaction, periodically checking which sstables are obsolete and simply
replacing them with the sstable that replaced it. These sstables are by definition only used
for lookup, since we are in the process of obsoleting the sstables we're compacting, they're
only used to reference overlapping ranges which may be covered by tombstones.
> A simplest solution might even be to simply detect obsoletion and recalculate our overlapping
tree afresh. This is a pretty quick operation in the grand scheme of things, certainly wrt
compaction, so nothing lost to do this at the rate we obsolete sstables.
> See CASSANDRA-7139 for original discussion of the problem.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message