incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arya Goudarzi <agouda...@gaiaonline.com>
Subject Re: COMMIT-LOG_WRITER Assertion Error
Date Tue, 10 Aug 2010 00:42:23 GMT
I've never run 0.6. I have been running of trunc with automatic svn update and build everyday
at 2pm. One of my nodes got this error which lead to the same last error prior to build and
restart today. Hope this helps better:

java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException:
java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.AssertionError
        at org.apache.cassandra.utils.FBUtilities.waitOnFutures(FBUtilities.java:549)
        at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:339)
        at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:174)
        at org.apache.cassandra.thrift.CassandraDaemon.setup(CassandraDaemon.java:120)
        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:90)
        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:224)
Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.RuntimeException:
java.util.concurrent.ExecutionException: java.lang.AssertionError
        at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
        at java.util.concurrent.FutureTask.get(FutureTask.java:111)
        at org.apache.cassandra.utils.FBUtilities.waitOnFutures(FBUtilities.java:545)
        ... 5 more
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: java.util.concurrent.ExecutionException:
java.lang.AssertionError
        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:636)
Caused by: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.AssertionError
        at org.apache.cassandra.db.commitlog.CommitLog.discardCompletedSegments(CommitLog.java:408)
        at org.apache.cassandra.db.ColumnFamilyStore$2.runMayThrow(ColumnFamilyStore.java:445)
        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
        ... 6 more
Caused by: java.util.concurrent.ExecutionException: java.lang.AssertionError
        at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
        at java.util.concurrent.FutureTask.get(FutureTask.java:111)
        at org.apache.cassandra.db.commitlog.CommitLog.discardCompletedSegments(CommitLog.java:400)
        ... 8 more
Caused by: java.lang.AssertionError
        at org.apache.cassandra.db.commitlog.CommitLogHeader$CommitLogHeaderSerializer.serialize(CommitLogHeader.java:157)
        at org.apache.cassandra.db.commitlog.CommitLogHeader.writeCommitLogHeader(CommitLogHeader.java:124)
        at org.apache.cassandra.db.commitlog.CommitLogSegment.writeHeader(CommitLogSegment.java:70)
        at org.apache.cassandra.db.commitlog.CommitLog.discardCompletedSegmentsInternal(CommitLog.java:450)
        at org.apache.cassandra.db.commitlog.CommitLog.access$300(CommitLog.java:75)
        at org.apache.cassandra.db.commitlog.CommitLog$6.call(CommitLog.java:394)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at org.apache.cassandra.db.commitlog.PeriodicCommitLogExecutorService$1.runMayThrow(PeriodicCommitLogExecutorService.java:52)
        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
        ... 1 more 

----- Original Message -----
From: "Jonathan Ellis" <jbellis@gmail.com>
To: user@cassandra.apache.org
Sent: Monday, August 9, 2010 5:18:35 PM
Subject: Re: COMMIT-LOG_WRITER Assertion Error

Sounds like you upgraded to trunk from 0.6 without draining your
commitlog first?

On Mon, Aug 9, 2010 at 3:30 PM, Arya Goudarzi <agoudarzi@gaiaonline.com> wrote:
> Just throwing this out there as it could be a concern. I had a cluster of 3 nodes running.
Over the weekend I updated to trunc (Aug 9th @ 2pm). Today, I came to run my daily tests and
my client kept giving me TSocket timeouts. Checking the error log of Cassandra servers, all
3 nodes had this and they all became unresponsive! Not sure how to reproduce this but a restart
of all 3 nodes fixed the issue:
>
> ERROR [COMMIT-LOG-WRITER] 2010-08-09 11:30:27,722 CassandraDaemon.java (line 82) Uncaught
exception in thread Thread[COMMIT-LOG-WRITER,5,main]
> java.lang.AssertionError
>        at org.apache.cassandra.db.commitlog.CommitLogHeader$CommitLogHeaderSerializer.serialize(CommitLogHeader.java:157)
>        at org.apache.cassandra.db.commitlog.CommitLogHeader.writeCommitLogHeader(CommitLogHeader.java:124)
>        at org.apache.cassandra.db.commitlog.CommitLogSegment.writeHeader(CommitLogSegment.java:70)
>        at org.apache.cassandra.db.commitlog.CommitLogSegment.write(CommitLogSegment.java:103)
>        at org.apache.cassandra.db.commitlog.CommitLog$LogRecordAdder.run(CommitLog.java:521)
>        at org.apache.cassandra.db.commitlog.PeriodicCommitLogExecutorService$1.runMayThrow(PeriodicCommitLogExecutorService.java:52)
>        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
>        at java.lang.Thread.run(Thread.java:636)
>
>
> -Arya
>



-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of Riptano, the source for professional Cassandra support
http://riptano.com

Mime
View raw message