cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Carlos Alonso <i...@mrcalonso.com>
Subject Re: Stale node decommission?
Date Thu, 08 Oct 2015 17:13:39 GMT
I ended up killing the node (nodetool removenode) because the streaming was
not working (probably because of the different sstables versions)

These guys were appearing in logs:

ERROR [STREAM-IN-/XXX.XXX.XXX.XXX] 2015-10-08 12:52:43,202
 StreamSession.java:505 - [Stream #75fd3ab0-6dbb-11e5-a275-8bc1b1cfbf0a]
Streaming error occurred

java.io.IOException: Connection reset by peer

    at sun.nio.ch.FileDispatcherImpl.read0(Native Method) ~[na:1.7.0_75]

    at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
~[na:1.7.0_75]

    at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
~[na:1.7.0_75]

    at sun.nio.ch.IOUtil.read(IOUtil.java:197) ~[na:1.7.0_75]

    at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
~[na:1.7.0_75]

    at
org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:51)
~[cassandra-all-2.1.9.791.jar:2.1.9.791]

    at
org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:250)
~[cassandra-all-2.1.9.791.jar:2.1.9.791]

    at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]

Carlos Alonso | Software Engineer | @calonso <https://twitter.com/calonso>

On 8 October 2015 at 17:31, Robert Coli <rcoli@eventbrite.com> wrote:

> On Thu, Oct 8, 2015 at 5:20 AM, Carlos Alonso <info@mrcalonso.com> wrote:
>
>> Hi guys, I’m decommissioning a node and the nodetool netstats bytes
>> count doesn’t move, have you seen it before?
>>
>
> Your decom stream is hung, if you don't have a version with the new
> non-infinite timeout for streams, you will have to restart the node and
> restart the decommission.
>
> =Rob
>

Mime
View raw message