cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mina Naguib (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-4411) Assertion with LCS compaction
Date Thu, 26 Jul 2012 14:34:35 GMT


Mina Naguib commented on CASSANDRA-4411:

Quick follow-up

All the problematic nodes have been offline scrubbed (successfully using --manifest-check
to speed things up).  There are no more compaction errors / pending compactions.

Like Anton, I'm a bit weary and keeping an eye on things - but so far so good.

On a tangent, it occurred to me that the amount of time it takes to run ( sstablescrub --manifest-check
) is mostly reading the sstables - the check itself and demoting the bad sstables to L0 appears
very cheap - would it be a good idea to perform that check automatically on cassandra startup
(after the sstables have been read) ?  It *may* be a quick fix for 1.1.3 to help people out
who have been bitten by this but don't know it yet.

> Assertion with LCS compaction
> -----------------------------
>                 Key: CASSANDRA-4411
>                 URL:
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.2
>            Reporter: Anton Winter
>            Assignee: Sylvain Lebresne
>             Fix For: 1.1.3
>         Attachments: 0001-Add-debugging-info-for-LCS.txt, 4411-followup.txt, 4411.txt,
assertion-w-more-debugging-info-omid.log, assertion.moreinfo.system.log, system.log
> As instructed in CASSANDRA-4321 I have raised this issue as a continuation of that issue
as it appears the problem still exists.
> I have repeatedly run sstablescrub across all my nodes after the 1.1.2 upgrade until
sstablescrub shows no errors.  The exceptions described in CASSANDRA-4321 do not occur as
frequently now but the integrity check still throws exceptions on a number of nodes.  Once
those exceptions occur compactionstats shows a large number of pending tasks with no progression
> {code}
> ERROR [CompactionExecutor:150] 2012-07-05 04:26:15,570 (line
134) Exception in thread Thread[CompactionExecutor:150,1,main]
> java.lang.AssertionError
>         at org.apache.cassandra.db.compaction.LeveledManifest.promote(
>         at org.apache.cassandra.db.compaction.LeveledCompactionStrategy.handleNotification(
>         at org.apache.cassandra.db.DataTracker.notifySSTablesChanged(
>         at org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(
>         at org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(
>         at org.apache.cassandra.db.compaction.CompactionTask.execute(
>         at org.apache.cassandra.db.compaction.LeveledCompactionTask.execute(
>         at org.apache.cassandra.db.compaction.CompactionManager$1.runMayThrow(
>         at
>         at java.util.concurrent.Executors$
>         at java.util.concurrent.FutureTask$Sync.innerRun(
>         at
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(
>         at java.util.concurrent.ThreadPoolExecutor$
>         at
> {code}

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message