cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benedict (JIRA)" <>
Subject [jira] [Updated] (CASSANDRA-7819) In progress compactions should not prevent deletion of stale sstables
Date Fri, 22 Aug 2014 09:38:11 GMT


Benedict updated CASSANDRA-7819:

    Attachment: 7819.txt

Went ahead and did it, since it was surprisingly simple to throw together. [~krummas] wdyt?
Not sure if our unit tests are sufficient to check the safety of it, but it seems trivially
safe enough to me, since compaction is single threaded.

> In progress compactions should not prevent deletion of stale sstables
> ---------------------------------------------------------------------
>                 Key: CASSANDRA-7819
>                 URL:
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Benedict
>            Priority: Minor
>              Labels: compaction
>             Fix For: 2.0.10
>         Attachments: 7819.txt
> Compactions retain references to potentially many sstables that existed when they were
started but that are now obsolete; many concurrent compactions can compound this dramatically,
and with very large files in size tiered compaction it is possible to inflate disk utilisation
dramatically beyond what is necessary.
> I propose, during compaction, periodically checking which sstables are obsolete and simply
replacing them with the sstable that replaced it. These sstables are by definition only used
for lookup, since we are in the process of obsoleting the sstables we're compacting, they're
only used to reference overlapping ranges which may be covered by tombstones.
> A simplest solution might even be to simply detect obsoletion and recalculate our overlapping
tree afresh. This is a pretty quick operation in the grand scheme of things, certainly wrt
compaction, so nothing lost to do this at the rate we obsolete sstables.
> See CASSANDRA-7139 for original discussion of the problem.

This message was sent by Atlassian JIRA

View raw message