cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Philippe <watche...@gmail.com>
Subject Re: Dealing with "Corrupt (negative) value length encountered"
Date Fri, 06 Jan 2012 07:24:18 GMT
Thanks Aaron, I was able to complete the repair by scrubbing the column
family on all three replicas.

Cheers

2012/1/4 aaron morton <aaron@thelastpickle.com>

> I was able to scrub the node the repair that failed was running on. Are
> you saying the error could be displayed on that node but the bad data
> coming from another node ?
>
> Yes. The error occurred the node was receiving a data stream from another,
> you will need to clean the source of the data. You can either crawl through
> the logs or scrub the entire cluster.
>
> Cheers
>
> -----------------
> Aaron Morton
> Freelance Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 4/01/2012, at 9:15 AM, Philippe wrote:
>
> I was able to scrub the node the repair that failed was running on. Are
> you saying the error could be displayed on that node but the bad data
> coming from another node ?
>
> Log inspection also showed many of these, they seem to happen around when
> a stream transfer finishes.
> ERROR [Thread-550876] 2012-01-03 16:35:31,922
> AbstractCassandraDaemon.java (line 139) Fatal exception in thread
> Thread[Thread-550876,5,main]
> java.lang.IllegalArgumentException
>         at
> sun.nio.ch.FileChannelImpl.transferFrom(FileChannelImpl.java:586)
>         at
> org.apache.cassandra.streaming.IncomingStreamReader.readnwrite(IncomingStreamReader.java:110)
>         at
> org.apache.cassandra.streaming.IncomingStreamReader.readFile(IncomingStreamReader.java:85)
>         at
> org.apache.cassandra.streaming.IncomingStreamReader.read(IncomingStreamReader.java:61)
>         at
> org.apache.cassandra.net.IncomingTcpConnection.stream(IncomingTcpConnection.java:189)
>         at
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:117)
>
> Thanks
>
> 2012/1/2 aaron morton <aaron@thelastpickle.com>
>
>> I would try to nodetool scrub the data on the node that that sent the bad
>> data in the stream. You may be able to work which node from the logs, or it
>> may be easier to just scrub them all.
>>
>> Hope that helps.
>>
>>   -----------------
>> Aaron Morton
>> Freelance Developer
>> @aaronmorton
>> http://www.thelastpickle.com
>>
>> On 31/12/2011, at 12:20 AM, Philippe wrote:
>>
>> Hello,
>> Running a combination of 0.8.6 and 0.8.8 with RF=3, I am getting the
>> following while repairing one node (all other nodes completed successfully).
>> Can I just stop the instance, erase the SSTable and restart cleanup ?
>> Thanks
>>
>> ERROR [Thread-402484] 2011-12-29 14:51:03,687
>> AbstractCassandraDaemon.java (line 139) Fatal exception in thread
>> Thread[Thread-402484,5,main]
>> java.lang.RuntimeException: java.util.concurrent.ExecutionException:
>> java.io.IOError: java.io.IOException: Corrupt (negative) value length
>> encountered
>>         at
>> org.apache.cassandra.streaming.StreamInSession.closeIfFinished(StreamInSession.java:154)
>>         at
>> org.apache.cassandra.streaming.IncomingStreamReader.read(IncomingStreamReader.java:63)
>>         at
>> org.apache.cassandra.net.IncomingTcpConnection.stream(IncomingTcpConnection.java:189)
>>         at
>> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:117)
>> Caused by: java.util.concurrent.ExecutionException: java.io.IOError:
>> java.io.IOException: Corrupt (negative) value length encountered
>>         at
>> java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>>         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>>         at
>> org.apache.cassandra.streaming.StreamInSession.closeIfFinished(StreamInSession.java:138)
>>         ... 3 more
>>
>>
>>
>
>

Mime
View raw message