cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daning Wang <dan...@netseer.com>
Subject Re: Cassandra Exception
Date Wed, 21 Mar 2012 17:27:17 GMT
and we are on 0.8.6.



On Wed, Mar 21, 2012 at 10:24 AM, Daning Wang <daning@netseer.com> wrote:

> Hi All,
>
>
> We got lots of Exception in the log, and later the server crashed. any
> idea what is happening and how to fix it?
>
> ERROR [RequestResponseStage:4] 2012-03-21 04:16:30,482
> AbstractCassandraDaemon.java (line 139) Fatal exception in thread
> Thread[RequestResponseStage:4,5,main]
> java.io.IOError: java.io.EOFException
>         at
> org.apache.cassandra.service.AbstractRowResolver.preprocess(AbstractRowResolver.java:71)
>         at
> org.apache.cassandra.service.ReadCallback.response(ReadCallback.java:125)
>         at
> org.apache.cassandra.net.ResponseVerbHandler.doVerb(ResponseVerbHandler.java:49)
>         at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:59)
>         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:722)
> Caused by: java.io.EOFException
>         at java.io.DataInputStream.readFully(DataInputStream.java:197)
>         at
> org.apache.cassandra.db.ReadResponseSerializer.deserialize(ReadResponse.java:104)
>         at
> org.apache.cassandra.db.ReadResponseSerializer.deserialize(ReadResponse.java:82)
>         at
> org.apache.cassandra.service.AbstractRowResolver.preprocess(AbstractRowResolver.java:64)
>         ... 6 more
> ERROR [RequestResponseStage:2] 2012-03-21 04:16:30,480
> AbstractCassandraDaemon.java (line 139) Fatal exception in thread
> Thread[RequestResponseStage:2,5,main]
> java.io.IOError:
> org.apache.cassandra.db.UnserializableColumnFamilyException: Couldn't find
> cfId=-387130991
>         at
> org.apache.cassandra.service.AbstractRowResolver.preprocess(AbstractRowResolver.java:71)
>         at
> org.apache.cassandra.service.AsyncRepairCallback.response(AsyncRepairCallback.java:47)
>         at
> org.apache.cassandra.net.ResponseVerbHandler.doVerb(ResponseVerbHandler.java:49)
>         at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:59)
>         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:722)
> Caused by: org.apache.cassandra.db.UnserializableColumnFamilyException:
> Couldn't find cfId=-387130991
>         at
> org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:123)
>         at org.apache.cassandra.db.RowSerializer.deserialize(Row.java:69)
>         at
> org.apache.cassandra.db.ReadResponseSerializer.deserialize(ReadResponse.java:113)
>         at
> org.apache.cassandra.db.ReadResponseSerializer.deserialize(ReadResponse.java:82)
>         at
> org.apache.cassandra.service.AbstractRowResolver.preprocess(AbstractRowResolver.java:64
>
>
>
> The is the exception before server crashes.
>
>
> ERROR [ReadRepairStage:299] 2012-03-21 05:02:53,808
> AbstractCassandraDaemon.java (line 139) Fatal exception in thread
> Thread[ReadRepairStage:299,5,main]
> java.lang.RuntimeException:
> java.util.concurrent.RejectedExecutionException: ThreadPoolExecutor has
> shut down
>         at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
>         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:722)
> Caused by: java.util.concurrent.RejectedExecutionException:
> ThreadPoolExecutor has shut down
>         at
> org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:60)
>         at
> java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:816)
>         at
> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1337)
>         at
> org.apache.cassandra.net.MessagingService.receive(MessagingService.java:490)
>         at
> org.apache.cassandra.net.MessagingService.sendOneWay(MessagingService.java:388)
>         at
> org.apache.cassandra.net.MessagingService.sendOneWay(MessagingService.java:346)
>         at
> org.apache.cassandra.service.RowRepairResolver.maybeScheduleRepairs(RowRepairResolver.java:121)
>         at
> org.apache.cassandra.service.RowRepairResolver.resolve(RowRepairResolver.java:85)
>         at
> org.apache.cassandra.service.AsyncRepairCallback$1.runMayThrow(AsyncRepairCallback.java:54)
>         at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
>         ... 3 more
>
> Thank you in advance,
>
> Daning
>

Mime
View raw message