cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Theroux (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-4456) AssertionError in ColumnFamilyStore.getOverlappingSSTables() during repair
Date Sat, 21 Jul 2012 19:15:34 GMT


Michael Theroux commented on CASSANDRA-4456:

I just hit this problem myself today, on a single node in a six node cluster.  I was running
nodetool repair, and it halted with this exception in the log.  I was monitoring the repair
pretty closely.  A couple of observations:

1) It happened while compaction of the same column family was happening simultaneously
2) When I re-ran it, it worked.

Note: I am not a cassandra developer, but I looked at the code.  A highly uneducated guess
is that an sstable was compacted and deleted while validation was expecting it to be there?
> AssertionError in ColumnFamilyStore.getOverlappingSSTables() during repair
> --------------------------------------------------------------------------
>                 Key: CASSANDRA-4456
>                 URL:
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.2
>         Environment: Ubuntu 11.04 64-bit
>            Reporter: Mike Heffner
> We have hit the following exception on several nodes while running repairs across our
1.1.2 ring. We've observed it happen on either the node executing the repair or a participating
replica in the repair operation. The result in either case is that the repair hangs.
> ERROR [ValidationExecutor:9] 2012-07-21 01:54:03,019 (line
134) Exception in thread Thread[ValidationExecutor:9,1,main]
> java.lang.AssertionError
>         at org.apache.cassandra.db.ColumnFamilyStore.getOverlappingSSTables(
>         at org.apache.cassandra.db.compaction.CompactionController.<init>(
>         at org.apache.cassandra.db.compaction.CompactionManager$ValidationCompactionIterable.<init>(
>         at org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(
>         at org.apache.cassandra.db.compaction.CompactionManager.access$600(
>         at org.apache.cassandra.db.compaction.CompactionManager$
>         at java.util.concurrent.FutureTask$Sync.innerRun(
>         at
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(
>         at java.util.concurrent.ThreadPoolExecutor$
>         at
> In building this ring we migrated sstables from an identical 0.8.8 ring by:
>  1. Creating the schema on our new 1.1.2 ring.
>  2. Rsyncing over sstables from 0.8.8 ring.
>  3. Renaming the sstables to match the directory and file naming structure of 1.1.x.
>  4. Ran nodetool refresh <keyspace> <cf> for each CF across each node.
>  5. Ran nodetool upgradesstables for each CF across each node.
> When those steps had completed, we began rolling repairs. Not all of the repair operations
have hit the exception -- some of the repairs have completed successfully.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message