cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lerh Chuan Low (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-13068) Fully expired sstable not dropped when running out of disk space
Date Tue, 30 May 2017 07:48:04 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-13068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16028817#comment-16028817
] 

Lerh Chuan Low commented on CASSANDRA-13068:
--------------------------------------------

Hi [~krummas],

Here's the new branch: https://github.com/apache/cassandra/compare/trunk...juiceblender:cassandra-13068

I've added some 3 different tests using the example, but I've also added an extra check 
{code}
final Set<SSTableReader> fullyExpiredSSTables = controller.getFullyExpiredSSTables();
            final Set<SSTableReader> actuallyCompact = Sets.difference(transaction.originals(),
fullyExpiredSSTables);

            // note that we need to do a rough estimate early if we can fit the compaction
on disk - this is pessimistic, but
            // since we might remove sstables from the compaction in checkAvailableDiskSpace
it needs to be done here
            // If there are no fully expired SSTables, check available disk space. Otherwise,
we want to try and always compact
            // fully expired SSTables
            if (fullyExpiredSSTables.isEmpty())
                checkAvailableDiskSpace(actuallyCompact);
{code}

This is to make the test I wrote {{testExpiredSSTablesStillGetDroppedWithNoDiskSpace()}} work.
I am not sure if this is the right way to do it or if it was intentionally left that way -
basically if you have expired SSTables and SSTables that are too big, we will throw the run
time exception instead of try and compact the expired SSTables away. I don't know if there's
ordering of compactions at this point though, so I don't know if this is a safe operation
to do. 

Feel free to let me know about any feedback. There seem to be other ways to go about this,
such as {{checkAvailableDiskSpace}} should just estimate fully expired SSTables to have compacted
file size 0 instead...? 

> Fully expired sstable not dropped when running out of disk space
> ----------------------------------------------------------------
>
>                 Key: CASSANDRA-13068
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-13068
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Compaction
>            Reporter: Marcus Eriksson
>            Assignee: Lerh Chuan Low
>              Labels: lhf
>             Fix For: 3.0.x, 3.11.x, 4.x
>
>
> If a fully expired sstable is larger than the remaining disk space we won't run the compaction
that can drop the sstable (ie, in our disk space check should not include the fully expired
sstables)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org


Mime
View raw message