cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jon Haddad <...@jonhaddad.com>
Subject Re: Streaming Failed during bootstrap of a Replacement node
Date Fri, 20 Dec 2019 19:13:04 GMT
You should upgrade to Cassandra 3.11.5 before doing anything else.  You're
running a pretty old and buggy version.  There's been hundreds (maybe
thousands) of bugs fixed between 3.3 and 3.11.5.

On Fri, Dec 20, 2019 at 10:46 AM Nethi, Manoj <Manoj.Nethi@fmr.com.invalid>
wrote:

> Hi,
>
>
>
> We are seeing the following error while bootstrapping a node which is
> replacement of a failed node in a multi DC cluster.
>
>
>
>
>
>
>
> WARN  [STREAM-IN-/**.***.***.**] 2019-12-19 23:36:40,120
> StreamSession.java:641 - [Stream #f231db30-22da-11ea-b38a-0f6cfab62953]
> Retrying for following error
>
> java.lang.ArrayIndexOutOfBoundsException: -127
>
>         at
> org.apache.cassandra.db.RangeTombstone$Bound$Serializer.deserialize(RangeTombstone.java:201)
> ~[apache-cassandra-3.3.0.jar:3.3.0]
>
>         at
> org.apache.cassandra.db.rows.UnfilteredSerializer.deserialize(UnfilteredSerializer.java:355)
> ~[apache-cassandra-3.3.0.jar:3.3.0]
>
>         at
> org.apache.cassandra.io.sstable.SSTableSimpleIterator$CurrentFormatIterator.computeNext(SSTableSimpleIterator.java:86)
> ~[apache-cassandra-3.3.0.jar:3.3.0]
>
>         at
> org.apache.cassandra.io.sstable.SSTableSimpleIterator$CurrentFormatIterator.computeNext(SSTableSimpleIterator.java:64)
> ~[apache-cassandra-3.3.0.jar:3.3.0]
>
>         at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> ~[apache-cassandra-3.3.0.jar:3.3.0]
>
>         at
> org.apache.cassandra.streaming.StreamReader$StreamDeserializer.hasNext(StreamReader.java:253)
> ~[apache-cassandra-3.3.0.jar:3.3.0]
>
>         at
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:108)
> ~[apache-cassandra-3.3.0.jar:3.3.0]
>
>         at
> org.apache.cassandra.db.ColumnIndex$Builder.build(ColumnIndex.java:120)
> ~[apache-cassandra-3.3.0.jar:3.3.0]
>
>         at
> org.apache.cassandra.db.ColumnIndex.writeAndBuildIndex(ColumnIndex.java:57)
> ~[apache-cassandra-3.3.0.jar:3.3.0]
>
>         at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:153)
> ~[apache-cassandra-3.3.0.jar:3.3.0]
>
>         at
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.append(SimpleSSTableMultiWriter.java:48)
> ~[apache-cassandra-3.3.0.jar:3.3.0]
>
>         at
> org.apache.cassandra.io.sstable.format.RangeAwareSSTableWriter.append(RangeAwareSSTableWriter.java:99)
> ~[apache-cassandra-3.3.0.jar:3.3.0]
>
>         at
> org.apache.cassandra.streaming.StreamReader.writePartition(StreamReader.java:178)
> ~[apache-cassandra-3.3.0.jar:3.3.0]
>
>         at
> org.apache.cassandra.streaming.compress.CompressedStreamReader.read(CompressedStreamReader.java:106)
> ~[apache-cassandra-3.3.0.jar:3.3.0]
>
>         at
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:50)
> [apache-cassandra-3.3.0.jar:3.3.0]
>
>         at
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:39)
> [apache-cassandra-3.3.0.jar:3.3.0]
>
>         at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:59)
> [apache-cassandra-3.3.0.jar:3.3.0]
>
>         at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
> [apache-cassandra-3.3.0.jar:3.3.0]
>
>         at java.lang.Thread.run(Thread.java:745) [na:1.8.0_74]
>
> ERROR [STREAM-IN-/**.***.***.**] 2019-12-19 23:36:40,120
> StreamSession.java:520 - [Stream #f231db30-22da-11ea-b38a-0f6cfab62953]
> Streaming error occurred
>
> java.lang.IllegalArgumentException: Unknown type 0
>
>         at
> org.apache.cassandra.streaming.messages.StreamMessage$Type.get(StreamMessage.java:97)
> ~[apache-cassandra-3.3.0.jar:3.3.0]
>
>         at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:58)
> ~[apache-cassandra-3.3.0.jar:3.3.0]
>
>         at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261)
> ~[apache-cassandra-3.3.0.jar:3.3.0]
>
>         at java.lang.Thread.run(Thread.java:745) [na:1.8.0_74]
>
> INFO  [STREAM-IN-/**.***.***.**] 2019-12-19 23:36:40,124
> StreamResultFuture.java:185 - [Stream
> #f231db30-22da-11ea-b38a-0f6cfab62953] Session with /**.***.**.** is
> complete
>
> WARN  [STREAM-IN-/**.***.***.**] 2019-12-19 23:36:40,124
> StreamResultFuture.java:212 - [Stream
> #f231db30-22da-11ea-b38a-0f6cfab62953] Stream failed
>
> WARN  [Thread-738] 2019-12-19 23:36:40,126 CompressedInputStream.java:182
> - Error while reading compressed input stream.
>
> java.nio.channels.ClosedChannelException: null
>
>         at
> sun.nio.ch.SocketChannelImpl.ensureReadOpen(SocketChannelImpl.java:257)
> ~[na:1.8.0_74]
>
>         at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:300)
> ~[na:1.8.0_74]
>
>         at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:59)
> ~[na:1.8.0_74]
>
>         at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:109)
> ~[na:1.8.0_74]
>
>         at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
> ~[na:1.8.0_74]
>
>         at
> org.apache.cassandra.streaming.compress.CompressedInputStream$Reader.runMayThrow(CompressedInputStream.java:172)
> ~[apache-cassandra-3.3.0.jar:3.3.0]
>
>         at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> [apache-cassandra-3.3.0.jar:3.3.0]
>
>         at java.lang.Thread.run(Thread.java:745) [na:1.8.0_74]
>
>
>
>
>
>
>
> Did any one faced a similar issue ? What should be done to resolve this ?
>
>
>
>
>
>
>
> Thanks
>
> Manoj
>

Mime
View raw message