I've never run 0.6. I have been running of trunc with automatic svn update and build everyday
at 2pm. One of my nodes got this error which lead to the same last error prior to build and
restart today. Hope this helps better:
java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException:
java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.AssertionError
at org.apache.cassandra.utils.FBUtilities.waitOnFutures(FBUtilities.java:549)
at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:339)
at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:174)
at org.apache.cassandra.thrift.CassandraDaemon.setup(CassandraDaemon.java:120)
at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:90)
at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:224)
Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.RuntimeException:
java.util.concurrent.ExecutionException: java.lang.AssertionError
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
at java.util.concurrent.FutureTask.get(FutureTask.java:111)
at org.apache.cassandra.utils.FBUtilities.waitOnFutures(FBUtilities.java:545)
... 5 more
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: java.util.concurrent.ExecutionException:
java.lang.AssertionError
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
Caused by: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.AssertionError
at org.apache.cassandra.db.commitlog.CommitLog.discardCompletedSegments(CommitLog.java:408)
at org.apache.cassandra.db.ColumnFamilyStore$2.runMayThrow(ColumnFamilyStore.java:445)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
... 6 more
Caused by: java.util.concurrent.ExecutionException: java.lang.AssertionError
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
at java.util.concurrent.FutureTask.get(FutureTask.java:111)
at org.apache.cassandra.db.commitlog.CommitLog.discardCompletedSegments(CommitLog.java:400)
... 8 more
Caused by: java.lang.AssertionError
at org.apache.cassandra.db.commitlog.CommitLogHeader$CommitLogHeaderSerializer.serialize(CommitLogHeader.java:157)
at org.apache.cassandra.db.commitlog.CommitLogHeader.writeCommitLogHeader(CommitLogHeader.java:124)
at org.apache.cassandra.db.commitlog.CommitLogSegment.writeHeader(CommitLogSegment.java:70)
at org.apache.cassandra.db.commitlog.CommitLog.discardCompletedSegmentsInternal(CommitLog.java:450)
at org.apache.cassandra.db.commitlog.CommitLog.access$300(CommitLog.java:75)
at org.apache.cassandra.db.commitlog.CommitLog$6.call(CommitLog.java:394)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at org.apache.cassandra.db.commitlog.PeriodicCommitLogExecutorService$1.runMayThrow(PeriodicCommitLogExecutorService.java:52)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
... 1 more
----- Original Message -----
From: "Jonathan Ellis" <jbellis@gmail.com>
To: user@cassandra.apache.org
Sent: Monday, August 9, 2010 5:18:35 PM
Subject: Re: COMMIT-LOG_WRITER Assertion Error
Sounds like you upgraded to trunk from 0.6 without draining your
commitlog first?
On Mon, Aug 9, 2010 at 3:30 PM, Arya Goudarzi <agoudarzi@gaiaonline.com> wrote:
> Just throwing this out there as it could be a concern. I had a cluster of 3 nodes running.
Over the weekend I updated to trunc (Aug 9th @ 2pm). Today, I came to run my daily tests and
my client kept giving me TSocket timeouts. Checking the error log of Cassandra servers, all
3 nodes had this and they all became unresponsive! Not sure how to reproduce this but a restart
of all 3 nodes fixed the issue:
>
> ERROR [COMMIT-LOG-WRITER] 2010-08-09 11:30:27,722 CassandraDaemon.java (line 82) Uncaught
exception in thread Thread[COMMIT-LOG-WRITER,5,main]
> java.lang.AssertionError
> at org.apache.cassandra.db.commitlog.CommitLogHeader$CommitLogHeaderSerializer.serialize(CommitLogHeader.java:157)
> at org.apache.cassandra.db.commitlog.CommitLogHeader.writeCommitLogHeader(CommitLogHeader.java:124)
> at org.apache.cassandra.db.commitlog.CommitLogSegment.writeHeader(CommitLogSegment.java:70)
> at org.apache.cassandra.db.commitlog.CommitLogSegment.write(CommitLogSegment.java:103)
> at org.apache.cassandra.db.commitlog.CommitLog$LogRecordAdder.run(CommitLog.java:521)
> at org.apache.cassandra.db.commitlog.PeriodicCommitLogExecutorService$1.runMayThrow(PeriodicCommitLogExecutorService.java:52)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> at java.lang.Thread.run(Thread.java:636)
>
>
> -Arya
>
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of Riptano, the source for professional Cassandra support
http://riptano.com
|