Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (nike.apache.org: domain of mor.yuki@gmail.com designates
 209.85.215.41 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <CDC4B10B.2929B%Dean.Hiller@nrel.gov>
References: 
 <CAGM0Up9bS2Xz1rAXQAcatNiBiNi792fXHKkgDH9X0FEvuP2O3g@mail.gmail.com>
	<CDC4B10B.2929B%Dean.Hiller@nrel.gov>
Date: Fri, 24 May 2013 09:55:52 -0500
Message-ID: 
 <CAGM0Up_OTe3Dx=6+N_QBmWSAGehOqwx+Gpniqs17KBxiju7Yzw@mail.gmail.com>
Subject: Re: exception causes streaming to hang forever
From: Yuki Morishita <mor.yuki@gmail.com>
To: "user@cassandra.apache.org" <user@cassandra.apache.org>
Content-Type: text/plain; charset=UTF-8

hmm, I only can say it may caused by corrupt SSTable...
Stream hang on unexpected error was fixed in 1.2.5
(https://issues.apache.org/jira/browse/CASSANDRA-5229).

On Fri, May 24, 2013 at 6:56 AM, Hiller, Dean <Dean.Hiller@nrel.gov> wrote:
> The exception on that node was just this
>
> ERROR [Thread-6056] 2013-05-22 14:47:59,416 CassandraDaemon.java (line
> 132) Exception in thread Thread[Thread-6056,5,main]
> java.lang.IndexOutOfBoundsException
>         at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:75)
>         at
> org.apache.cassandra.streaming.compress.CompressedInputStream$Reader.runMay
> Throw(CompressedInputStream.java:151)
>         at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
>         at java.lang.Thread.run(Thread.java:662)
>
>
>
>
> On 5/23/13 9:51 AM, "Yuki Morishita" <mor.yuki@gmail.com> wrote:
>
>>What kind of error does the other end of streaming(/10.10.42.36) say?
>>
>>On Wed, May 22, 2013 at 5:19 PM, Hiller, Dean <Dean.Hiller@nrel.gov>
>>wrote:
>>> We had 3 nodes roll on good and the next 2, we see a remote node with
>>>this exception every time we start over and bootstrap the node
>>>
>>> ERROR [Streaming to /10.10.42.36:2] 2013-05-22 14:47:59,404
>>>CassandraDaemon.java (line 132) Exception in thread Thread[Streaming to
>>>/10.10.42.36:2,5,main]
>>> java.lang.RuntimeException: java.io.IOException: Input/output error
>>>         at
>>>com.google.common.base.Throwables.propagate(Throwables.java:160)
>>>         at
>>>org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:32)
>>>         at
>>>java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor
>>>.java:895)
>>>         at
>>>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.jav
>>>a:918)
>>>         at java.lang.Thread.run(Thread.java:662)
>>> Caused by: java.io.IOException: Input/output error
>>>         at sun.nio.ch.FileChannelImpl.transferTo0(Native Method)
>>>         at
>>>sun.nio.ch.FileChannelImpl.transferToDirectly(FileChannelImpl.java:405)
>>>         at
>>>sun.nio.ch.FileChannelImpl.transferTo(FileChannelImpl.java:506)
>>>         at
>>>org.apache.cassandra.streaming.compress.CompressedFileStreamTask.stream(C
>>>ompressedFileStreamTask.java:90)
>>>         at
>>>org.apache.cassandra.streaming.FileStreamTask.runMayThrow(FileStreamTask.
>>>java:91)
>>>         at
>>>org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
>>>         ... 3 more
>>>
>>> Are there any ideas what this is?  Google doesn't real show any useful
>>>advice on this and our node has not joined the ring yet so I don't think
>>>we can run a repair just yet to avoid it and try synching via another
>>>means.  It seems on a streaming failure, it never recovers from this.
>>>Any ideas?
>>>
>>> We are on cassandra 1.2.2
>>>
>>> Thanks,
>>> Dean
>>>
>>
>>
>>
>>--
>>Yuki Morishita
>> t:yukim (http://twitter.com/yukim)
>


-- 
Yuki Morishita
 t:yukim (http://twitter.com/yukim)