incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robin Verlangen <ro...@us2.nl>
Subject Re: Node crashing during read repair
Date Tue, 03 Jul 2012 13:25:51 GMT
Hi Aaron,

This was the first error. It occurred a couple of times after this. We did
an hardware upgrade on the server and increased the max heap size. Now
running fine. Seems that 1.1.1 uses a little more memory, or or data set
just grew ;-)

Thank you for your time!

2012/7/3 aaron morton <aaron@thelastpickle.com>

> Is this still an issue ?
>
> It looks like something shut down the messaging service. Was there
> anything else in the logs ?
>
> Cheers
>
> -----------------
> Aaron Morton
> Freelance Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 28/06/2012, at 3:49 AM, Robin Verlangen wrote:
>
> Hi there,
>
> Today I found one node (running 1.1.1 in a 3 node cluster) being dead for
> the third time this week, it died with the following message:
>
> ERROR [ReadRepairStage:3] 2012-06-27 14:28:30,929
> AbstractCassandraDaemon.java (line 134) Exception in thread
> Thread[ReadRepairStage:3,5,main]
> java.util.concurrent.RejectedExecutionException: ThreadPoolExecutor has
> shut down
>         at
> org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:60)
>         at java.util.concurrent.ThreadPoolExecutor.reject(Unknown Source)
>         at java.util.concurrent.ThreadPoolExecutor.execute(Unknown Source)
>         at
> org.apache.cassandra.net.MessagingService.receive(MessagingService.java:566)
>         at
> org.apache.cassandra.net.MessagingService.sendOneWay(MessagingService.java:439)
>         at
> org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:391)
>         at
> org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:372)
>         at
> org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:460)
>         at
> org.apache.cassandra.service.RowRepairResolver.scheduleRepairs(RowRepairResolver.java:136)
>         at
> org.apache.cassandra.service.RowRepairResolver.resolve(RowRepairResolver.java:94)
>         at
> org.apache.cassandra.service.AsyncRepairCallback$1.runMayThrow(AsyncRepairCallback.java:54)
>         at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
> Source)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
> Source)
>         at java.lang.Thread.run(Unknown Source)
>
> Is this a common bug in 1.1.1, or did I get a "race" condition?
>
> --
> With kind regards,
>
> Robin Verlangen
> *Software engineer*
> *
> *
> W http://www.robinverlangen.nl
> E robin@us2.nl
>
> Disclaimer: The information contained in this message and attachments is
> intended solely for the attention and use of the named addressee and may be
> confidential. If you are not the intended recipient, you are reminded that
> the information remains the property of the sender. You must not use,
> disclose, distribute, copy, print or rely on this e-mail. If you have
> received this message in error, please contact the sender immediately and
> irrevocably delete this message and any copies.
>
>
>


-- 
With kind regards,

Robin Verlangen
*Software engineer*
*
*
W http://www.robinverlangen.nl
E robin@us2.nl

Disclaimer: The information contained in this message and attachments is
intended solely for the attention and use of the named addressee and may be
confidential. If you are not the intended recipient, you are reminded that
the information remains the property of the sender. You must not use,
disclose, distribute, copy, print or rely on this e-mail. If you have
received this message in error, please contact the sender immediately and
irrevocably delete this message and any copies.

Mime
View raw message