cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nikolai Grigoriev (JIRA)" <>
Subject [jira] [Created] (CASSANDRA-8190) Compactions stop completely because of RuntimeException in CompactionExecutor
Date Sun, 26 Oct 2014 23:17:33 GMT
Nikolai Grigoriev created CASSANDRA-8190:

             Summary: Compactions stop completely because of RuntimeException in CompactionExecutor
                 Key: CASSANDRA-8190
             Project: Cassandra
          Issue Type: Bug
          Components: Core
         Environment: DSE 4.5.2 (Cassandra 2.0.10)
            Reporter: Nikolai Grigoriev

I have a cluster that is recovering from being overloaded with writes.  I am using the workaround
from CASSANDRA-6621 to prevent the STCS fallback (which is killing the cluster - see CASSANDRA-7949).

I have observed that after one or more exceptions like this

ERROR [CompactionExecutor:4087] 2014-10-26 22:50:05,016 (line 199) Exception
in thread Thread[CompactionExecutor:4087,1,main]
java.lang.RuntimeException: Last written key DecoratedKey(425124616570337476, 0010000000001111000000000000033523da00001000000000033523da000000001111000000001000000000
00004000000000000000000100) >= current key DecoratedKey(-8778432288598355336, 0010000000001111000000000000040c7a8f00001000000000040c7a8f000000001111000000001000000000
00004000000000000000000100) writing into /cassandra-data/disk2/myks/mytable/myks-mytable-tmp-jb-130379-Data.db
        at org.apache.cassandra.db.compaction.CompactionTask.runWith(
        at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(
        at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(
        at org.apache.cassandra.db.compaction.CompactionManager$
        at java.util.concurrent.Executors$
        at java.util.concurrent.ThreadPoolExecutor.runWorker(
        at java.util.concurrent.ThreadPoolExecutor$

the node completely stops the compactions and I end up in the state like this:

# nodetool compactionstats
pending tasks: 1288
          compaction type        keyspace           table       completed           total
     unit  progress
Active compaction remaining time :        n/a

The node recovers if restarted and starts compactions - until getting more exceptions like

This message was sent by Atlassian JIRA

View raw message