cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From aaron morton <aa...@thelastpickle.com>
Subject Re: Dealing with "Corrupt (negative) value length encountered"
Date Wed, 04 Jan 2012 08:18:51 GMT
> I was able to scrub the node the repair that failed was running on. Are you saying the
error could be displayed on that node but the bad data coming from another node ?
Yes. The error occurred the node was receiving a data stream from another, you will need to
clean the source of the data. You can either crawl through the logs or scrub the entire cluster.


Cheers
 
-----------------
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 4/01/2012, at 9:15 AM, Philippe wrote:

> I was able to scrub the node the repair that failed was running on. Are you saying the
error could be displayed on that node but the bad data coming from another node ?
> 
> Log inspection also showed many of these, they seem to happen around when a stream transfer
finishes.
> ERROR [Thread-550876] 2012-01-03 16:35:31,922 AbstractCassandraDaemon.java (line 139)
Fatal exception in thread Thread[Thread-550876,5,main]
> java.lang.IllegalArgumentException
>         at sun.nio.ch.FileChannelImpl.transferFrom(FileChannelImpl.java:586)
>         at org.apache.cassandra.streaming.IncomingStreamReader.readnwrite(IncomingStreamReader.java:110)
>         at org.apache.cassandra.streaming.IncomingStreamReader.readFile(IncomingStreamReader.java:85)
>         at org.apache.cassandra.streaming.IncomingStreamReader.read(IncomingStreamReader.java:61)
>         at org.apache.cassandra.net.IncomingTcpConnection.stream(IncomingTcpConnection.java:189)
>         at org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:117)
> 
> Thanks
> 
> 2012/1/2 aaron morton <aaron@thelastpickle.com>
> I would try to nodetool scrub the data on the node that that sent the bad data in the
stream. You may be able to work which node from the logs, or it may be easier to just scrub
them all. 
> 
> Hope that helps. 
>  
> -----------------
> Aaron Morton
> Freelance Developer
> @aaronmorton
> http://www.thelastpickle.com
> 
> On 31/12/2011, at 12:20 AM, Philippe wrote:
> 
>> Hello,
>> Running a combination of 0.8.6 and 0.8.8 with RF=3, I am getting the following while
repairing one node (all other nodes completed successfully).
>> Can I just stop the instance, erase the SSTable and restart cleanup ?
>> Thanks
>> 
>> ERROR [Thread-402484] 2011-12-29 14:51:03,687 AbstractCassandraDaemon.java (line
139) Fatal exception in thread Thread[Thread-402484,5,main]
>> java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.io.IOError:
java.io.IOException: Corrupt (negative) value length encountered
>>         at org.apache.cassandra.streaming.StreamInSession.closeIfFinished(StreamInSession.java:154)
>>         at org.apache.cassandra.streaming.IncomingStreamReader.read(IncomingStreamReader.java:63)
>>         at org.apache.cassandra.net.IncomingTcpConnection.stream(IncomingTcpConnection.java:189)
>>         at org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:117)
>> Caused by: java.util.concurrent.ExecutionException: java.io.IOError: java.io.IOException:
Corrupt (negative) value length encountered
>>         at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>>         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>>         at org.apache.cassandra.streaming.StreamInSession.closeIfFinished(StreamInSession.java:138)
>>         ... 3 more
>> 
> 
> 


Mime
View raw message