cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Philip Thompson (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (CASSANDRA-8665) Cassandra does not start with NPE in ColumnFamilyStore.removeUnfinishedCompactionLeftovers
Date Thu, 19 Mar 2015 14:38:50 GMT

     [ https://issues.apache.org/jira/browse/CASSANDRA-8665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Philip Thompson resolved CASSANDRA-8665.
----------------------------------------
       Resolution: Cannot Reproduce
    Fix Version/s:     (was: 2.1.4)
                   2.1.3

Okay, please re-open if it comes back up.

> Cassandra does not start with NPE in ColumnFamilyStore.removeUnfinishedCompactionLeftovers
> ------------------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-8665
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-8665
>             Project: Cassandra
>          Issue Type: Bug
>         Environment: Ubuntu 12.04 | C* 2.1.2 | ruby-driver 1.2
>            Reporter: Kishan Karunaratne
>             Fix For: 2.1.3
>
>
> During a ruby driver endurance/duration test, the following error occurred:
> {noformat}
> /mnt/systemlogs/system.log:ERROR [main] 2015-01-17 21:18:25,780 CassandraDaemon.java:482
- Exception encountered during startup
> /mnt/systemlogs/system.log-java.lang.NullPointerException: null
> /mnt/systemlogs/system.log-	at org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:573)
~[main/:na]
> /mnt/systemlogs/system.log-	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:249)
[main/:na]
> /mnt/systemlogs/system.log-	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:465)
[main/:na]
> /mnt/systemlogs/system.log-	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:554)
[main/:na]
> {noformat}
> Here is the system.log leading up to the error:
> {noformat}
> INFO  [main] 2015-01-17 21:18:24,581 ColumnFamilyStore.java:278 - Initializing system.peers
> INFO  [SSTableBatchOpen:1] 2015-01-17 21:18:24,593 SSTableReader.java:392 - Opening /srv/performance/cass/data/system/peers-37f71aca7dc2383ba70672528af04d4f/system-peers-ka-169
(10533 bytes)
> INFO  [SSTableBatchOpen:1] 2015-01-17 21:18:24,597 SSTableReader.java:392 - Opening /srv/performance/cass/data/system/peers-37f71aca7dc2383ba70672528af04d4f/system-peers-ka-171
(10572 bytes)
> INFO  [SSTableBatchOpen:1] 2015-01-17 21:18:24,598 SSTableReader.java:392 - Opening /srv/performance/cass/data/system/peers-37f71aca7dc2383ba70672528af04d4f/system-peers-ka-170
(10581 bytes)
> INFO  [main] 2015-01-17 21:18:24,609 ColumnFamilyStore.java:278 - Initializing system.local
> INFO  [SSTableBatchOpen:1] 2015-01-17 21:18:24,613 SSTableReader.java:392 - Opening /srv/performance/cass/data/system/local-7ad54392bcdd35a684174e047860b377/system-local-ka-679
(5257 bytes)
> INFO  [SSTableBatchOpen:1] 2015-01-17 21:18:24,616 SSTableReader.java:392 - Opening /srv/performance/cass/data/system/local-7ad54392bcdd35a684174e047860b377/system-local-ka-678
(5679 bytes)
> {noformat}
> Cassandra attempted to restart twice unsuccessfully (this error occurred twice) and then
gave up; it seems like a corrupt Data.db file? The endurance test consists of a chaos rhino
which randomly rolling restarts a node.
> The only other significant error is TombstoneOverwhelmingException and is probably unrelated:
> {noformat}
> /mnt/systemlogs/system.log:ERROR [HintedHandoff:2] 2015-01-17 12:46:32,378 SliceQueryFilter.java:218
- Scanned over 100000 tombstones in system.hints; query aborted (see tombstone_failure_threshold)
> /mnt/systemlogs/system.log:ERROR [HintedHandoff:2] 2015-01-17 12:46:32,416 CassandraDaemon.java:170
- Exception in thread Thread[HintedHandoff:2,1,main]
> /mnt/systemlogs/system.log-org.apache.cassandra.db.filter.TombstoneOverwhelmingException:
null
> /mnt/systemlogs/system.log-	at org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:220)
~[main/:na]
> /mnt/systemlogs/system.log-	at org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:107)
~[main/:na]
> /mnt/systemlogs/system.log-	at org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:81)
~[main/:na]
> /mnt/systemlogs/system.log-	at org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:69)
~[main/:na]
> /mnt/systemlogs/system.log-	at org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:320)
~[main/:na]
> /mnt/systemlogs/system.log-	at org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:62)
~[main/:na]
> /mnt/systemlogs/system.log-	at org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1885)
~[main/:na]
> /mnt/systemlogs/system.log-	at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1693)
~[main/:na]
> /mnt/systemlogs/system.log-	at org.apache.cassandra.db.HintedHandOffManager.doDeliverHintsToEndpoint(HintedHandOffManager.java:378)
~[main/:na]
> /mnt/systemlogs/system.log-	at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:337)
~[main/:na]
> /mnt/systemlogs/system.log-	at org.apache.cassandra.db.HintedHandOffManager.access$400(HintedHandOffManager.java:88)
~[main/:na]
> /mnt/systemlogs/system.log-	at org.apache.cassandra.db.HintedHandOffManager$5.run(HintedHandOffManager.java:548)
~[main/:na]
> /mnt/systemlogs/system.log-	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
~[na:1.7.0_25]
> /mnt/systemlogs/system.log-	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
~[na:1.7.0_25]
> /mnt/systemlogs/system.log-	at java.util.concurrent.FutureTask.run(FutureTask.java:166)
~[na:1.7.0_25]
> /mnt/systemlogs/system.log-	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
~[na:1.7.0_25]
> /mnt/systemlogs/system.log-	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
~[na:1.7.0_25]
> /mnt/systemlogs/system.log-	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
~[na:1.7.0_25]
> /mnt/systemlogs/system.log-	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
~[na:1.7.0_25]
> /mnt/systemlogs/system.log-	at java.lang.Thread.run(Thread.java:724) ~[na:1.7.0_25]
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message