cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sylvain Lebresne (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-3306) Error in LeveledCompactionStrategy
Date Wed, 31 Oct 2012 09:49:13 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-3306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487641#comment-13487641
] 

Sylvain Lebresne commented on CASSANDRA-3306:
---------------------------------------------

bq. Ok. I've fixed this and reattached 0002.

Alright, +1 on 0002. Let's commit that for now to 1.1/1.2 as this fix this ticket.

{quote}
the test assumes that SSTables are added to List in order of flush
And sstablesIterated counter in collectAllData is doing fine because we actually read the
data
{quote}

Right. I guess what I meant is that what is tested right now is not really sensible. Relying
on the order of flush is only valid for a small, controlled test, but in reality as soon as
compaction kicks in, the order of sstable in DataTracker will be meaningless even with a List
instead of a Set. Basically the guarantee collectAll gives us today is that it will eliminate
sstables whose maxTimestamp < mostRecentTombstone with just having read the sstable row
header, not the full data. But that's not what sstablesIterated counts so it's broken.

That being said, I think we can improve collectAll in the way described in CASSANDRA-4883.
If we do so, the test will pass again without relying on any assumption of the order of sstables
in DataTracker. So overall I suggest moving all of this to CASSANDRA-4883.
                
> Error in LeveledCompactionStrategy
> ----------------------------------
>
>                 Key: CASSANDRA-3306
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3306
>             Project: Cassandra
>          Issue Type: Bug
>    Affects Versions: 1.0.0
>            Reporter: Radim Kolar
>            Assignee: Yuki Morishita
>             Fix For: 1.1.7
>
>         Attachments: 0001-CASSANDRA-3306-test.patch, 0001-change-DataTracker.View-s-sstables-from-List-to-Set.patch,
0002-fail-stream-session-for-invalid-request.patch
>
>
> during stress testing, i always get this error making leveledcompaction strategy unusable.
Should be easy to reproduce - just write fast.
> ERROR [CompactionExecutor:6] 2011-10-04 15:48:52,179 AbstractCassandraDaemon.java (line
133) Fatal exception in thread Thread[CompactionExecutor:6,5,main]
> java.lang.AssertionError
> 	at org.apache.cassandra.db.DataTracker$View.newSSTables(DataTracker.java:580)
> 	at org.apache.cassandra.db.DataTracker$View.replace(DataTracker.java:546)
> 	at org.apache.cassandra.db.DataTracker.replace(DataTracker.java:268)
> 	at org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(DataTracker.java:232)
> 	at org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(ColumnFamilyStore.java:960)
> 	at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:199)
> 	at org.apache.cassandra.db.compaction.LeveledCompactionTask.execute(LeveledCompactionTask.java:47)
> 	at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:131)
> 	at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:114)
> 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> 	at java.lang.Thread.run(Thread.java:662)
> and this is in json data for table:
> {
>   "generations" : [ {
>     "generation" : 0,
>     "members" : [ 459, 460, 461, 462, 463, 464, 465, 466, 467, 468, 469, 470, 471, 472,
473, 474, 475, 476, 477, 478, 479, 480, 481, 482, 483, 484 ]
>   }, {
>     "generation" : 1,
>     "members" : [ ]
>   }, {
>     "generation" : 2,
>     "members" : [ ]
>   }, {
>     "generation" : 3,
>     "members" : [ ]
>   }, {
>     "generation" : 4,
>     "members" : [ ]
>   }, {
>     "generation" : 5,
>     "members" : [ ]
>   }, {
>     "generation" : 6,
>     "members" : [ ]
>   }, {
>     "generation" : 7,
>     "members" : [ ]
>   } ]
> }

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message