cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Maki Watanabe (Issue Comment Edited) (JIRA)" <j...@apache.org>
Subject [jira] [Issue Comment Edited] (CASSANDRA-3989) nodetool cleanup/scrub/upgradesstables promotes all sstables to next level (LeveledCompaction)
Date Wed, 07 Mar 2012 05:57:59 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13224001#comment-13224001
] 

Maki Watanabe edited comment on CASSANDRA-3989 at 3/7/12 5:56 AM:
------------------------------------------------------------------

It is more complicated than I first thought.
getCandidatesFor(int level) returns next compaction candidates for the level by:
- sort the generations and find a sstable where we left off last time
- and then find overlappng sstables from next level

So if we add a new sstable into same level, getCompactionCandidates won't return empty, and
then getNextBackgoundTask returns non-null task forever.
I think we should better to back out the fix and look for better strategy to resolve the issue.
                
      was (Author: makiw):
    It is more complicated than I first thought.
getCandidatesFor(int level) returns next compaction candidates for the level by:
- sor the generations and find a sstable where we left off last time
- and then find overlappng sstables from next level

So if we add a new sstable into same level, getCompactionCandidates won't return empty, and
then getNextBackgoundTask returns non-null task forever.
I think we should better to back out the fix and look for better strategy to resolve the issue.
                  
> nodetool cleanup/scrub/upgradesstables promotes all sstables to next level (LeveledCompaction)
> ----------------------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-3989
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3989
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.0.0
>         Environment: RHEL6
>            Reporter: Maki Watanabe
>            Assignee: Maki Watanabe
>            Priority: Minor
>             Fix For: 1.0.9, 1.1.0
>
>         Attachments: 0001-Fix-promote-not-to-promote-files-at-cleanup-compacti.patch
>
>
> 1.0.7 + LeveledCompactionStrategy
> If you run nodetool cleanup, scrub, or upgradesstables, Cassandra execute compaction
for each sstable. During the compaction, it put the new sstable to next level of the original
sstable. If you run cleanup many times, sstables will reached to the highest level, and CASSANDRA-3608
will happens at next cleanup.
> Reproduce procedure:
> # create column family CF1 with compaction_strategy=LeveledCompactionStrategy and compaction_strategy_options={sstable_size_in_mb:
5};
> # Insert some data into CF1.
> # nodetool flush
> # Verify the sstable is created at L1 in CF1.json
> # nodetool cleanup
> # Verify sstable in L1 is removed and new sstable is created at L2 in CF1.json
> # repeat nodetool cleanup some times

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message