cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron Morton <aa...@thelastpickle.com>
Subject 0.7.0.bet1 errors during start up
Date Thu, 26 Aug 2010 03:10:09 GMT
0.7.0-bet1 4 node clustered, i'd managed to get it into some sort of awful state (i think by
accidently creating to many clients, it was also complaining about running out of file handles). Anyway
I killed it all and restarted just one node, thought I would let it settle down then start
the others. On the first node I got this.

(Sorry I cannot be more specific, was not paying too much attention it it all went bang)

I managed to get a couple of errors, one of which shutdown the server. Just checking before
putting them into Jira, should I split them up? 

ERROR [pool-1-thread-29] 2010-08-26 14:58:20,021 Cassandra.java (line 2651) Internal error
processing get_slice
java.lang.IllegalStateException: replication factor (3) exceeds number of endpoints (1)
        at org.apache.cassandra.locator.RackUnawareStrategy.calculateNaturalEndpoints(RackUnawareStrategy.java:57)
        at org.apache.cassandra.locator.AbstractReplicationStrategy.getNaturalEndpoints(AbstractReplicationStrategy.java:88)
        at org.apache.cassandra.service.StorageService.getNaturalEndpoints(StorageService.java:1289)
        at org.apache.cassandra.service.StorageService.getNaturalEndpoints(StorageService.java:1277)
        at org.apache.cassandra.service.StorageService.findSuitableEndpoint(StorageService.java:1323)
        at org.apache.cassandra.service.StorageProxy.strongRead(StorageProxy.java:402)
        at org.apache.cassandraservice.StorageProxy.readProtocol(StorageProxy.java:302)
        at org.apache.cassandra.thrift.CassandraServer.readColumnFamily(CassandraServer.java:125)
        at org.apache.cassandra.thrift.CassandraServer.getSlice(CassandraServer.java:231)
        at org.apache.cassandra.thrift.CassandraServer.multigetSliceInternal(CassandraServer.java:309)
        at org.apache.cassandra.thrift.CassandraServer.get_slice(CassandraServer.java:270)
        at org.apache.cassandra.thrift.Cassandra$Processor$get_slice.process(Cassandra.java:2643)
        at org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.java:2499)
        at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:167)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)   


So I started the other 3, and two suffered the error below which caused the process to shutdown...

ERROR [main] 2010-08-26 14:59:22,315 AbstractCassandraDaemon.java (line 107) Exception encountered
during startup.
java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException:
java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.AssertionError
        at org.apache.cassandra.utils.FBUtilities.waitOnFutures(FBUtilities.java:549)
        at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:339)
        at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:174)
        at org.apache.cassandra.thrift.CassandraDaemon.setup(CassandraDaemon.java:120)
        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:90)
        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:224)
Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.RuntimeException:
java.util.concurrent.ExecutionException: java.lang.AssertionError
        at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
        at java.util.concurrent.FutureTask.get(FutureTask.java:83)
        at org.apache.cassandra.utils.FBUtilities.waitOnFutures(FBUtilities.java:545)
        ... 5 more
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: java.util.concurrent.ExecutionException:
java.lang.AssertionError
        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.AssertionError
        at org.apache.cassandra.db.commitlog.CommitLog.discardCompletedSegments(CommitLog.java:408)
        at org.apache.cassandra.db.ColumnFamilyStore$2.runMayThrow(ColumnFamilyStore.java:445)
        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
        ... 6 more
Caused by: java.util.concurrent.ExecutionException: java.lang.AssertionError
        at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
        at java.util.concurrent.FutureTask.get(FutureTask.java:83)
        at org.apachecassandra.db.commitlog.CommitLog.discardCompletedSegments(CommitLog.java:400)
        ... 8 more
Caused by: java.lang.AssertionError
        at org.apache.cassandra.db.commitlog.CommitLogHeader$CommitLogHeaderSerializer.serialize(CommitLogHeader.java:157)
        at org.apache.cassandra.db.commitlog.CommitLogHeader.writeCommitLogHeader(CommitLogHeader.java:124)
        at org.apache.cassandra.db.commitlog.CommitLogSegment.writeHeader(CommitLogSegment.java:70)
        at org.apache.cassandra.db.commitlog.CommitLog.discardCompletedSegmentsInternal(CommitLog.java:450)
        at org.apache.cassandra.db.commitlog.CommitLog.access$300(CommitLog.java:75)
        at org.apache.cassandra.db.commitlog.CommitLog$6.call(CommitLog.java:394)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at org.apache.cassandra.db.commitlog.PeriodicCommitLogExecutorService$1.runMayThrow(PeriodicCommitLogExecutorService.java:52)
        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
        ... 1 more


Aaron


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
    • Unnamed multipart/related (inline, None, 0 bytes)
View raw message