incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hiller, Dean" <Dean.Hil...@nrel.gov>
Subject Re: exception causes streaming to hang forever
Date Fri, 24 May 2013 15:06:14 GMT
Lol, yup, I just found out that is the issue!!!!
Thanks,
Dean

On 5/24/13 8:55 AM, "Yuki Morishita" <mor.yuki@gmail.com> wrote:

>hmm, I only can say it may caused by corrupt SSTable...
>Stream hang on unexpected error was fixed in 1.2.5
>(https://issues.apache.org/jira/browse/CASSANDRA-5229).
>
>On Fri, May 24, 2013 at 6:56 AM, Hiller, Dean <Dean.Hiller@nrel.gov>
>wrote:
>> The exception on that node was just this
>>
>> ERROR [Thread-6056] 2013-05-22 14:47:59,416 CassandraDaemon.java (line
>> 132) Exception in thread Thread[Thread-6056,5,main]
>> java.lang.IndexOutOfBoundsException
>>         at 
>>sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:75)
>>         at
>> 
>>org.apache.cassandra.streaming.compress.CompressedInputStream$Reader.runM
>>ay
>> Throw(CompressedInputStream.java:151)
>>         at
>> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
>>         at java.lang.Thread.run(Thread.java:662)
>>
>>
>>
>>
>> On 5/23/13 9:51 AM, "Yuki Morishita" <mor.yuki@gmail.com> wrote:
>>
>>>What kind of error does the other end of streaming(/10.10.42.36) say?
>>>
>>>On Wed, May 22, 2013 at 5:19 PM, Hiller, Dean <Dean.Hiller@nrel.gov>
>>>wrote:
>>>> We had 3 nodes roll on good and the next 2, we see a remote node with
>>>>this exception every time we start over and bootstrap the node
>>>>
>>>> ERROR [Streaming to /10.10.42.36:2] 2013-05-22 14:47:59,404
>>>>CassandraDaemon.java (line 132) Exception in thread Thread[Streaming to
>>>>/10.10.42.36:2,5,main]
>>>> java.lang.RuntimeException: java.io.IOException: Input/output error
>>>>         at
>>>>com.google.common.base.Throwables.propagate(Throwables.java:160)
>>>>         at
>>>>org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:32)
>>>>         at
>>>>java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecut
>>>>or
>>>>.java:895)
>>>>         at
>>>>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.j
>>>>av
>>>>a:918)
>>>>         at java.lang.Thread.run(Thread.java:662)
>>>> Caused by: java.io.IOException: Input/output error
>>>>         at sun.nio.ch.FileChannelImpl.transferTo0(Native Method)
>>>>         at
>>>>sun.nio.ch.FileChannelImpl.transferToDirectly(FileChannelImpl.java:405)
>>>>         at
>>>>sun.nio.ch.FileChannelImpl.transferTo(FileChannelImpl.java:506)
>>>>         at
>>>>org.apache.cassandra.streaming.compress.CompressedFileStreamTask.stream
>>>>(C
>>>>ompressedFileStreamTask.java:90)
>>>>         at
>>>>org.apache.cassandra.streaming.FileStreamTask.runMayThrow(FileStreamTas
>>>>k.
>>>>java:91)
>>>>         at
>>>>org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
>>>>         ... 3 more
>>>>
>>>> Are there any ideas what this is?  Google doesn't real show any useful
>>>>advice on this and our node has not joined the ring yet so I don't
>>>>think
>>>>we can run a repair just yet to avoid it and try synching via another
>>>>means.  It seems on a streaming failure, it never recovers from this.
>>>>Any ideas?
>>>>
>>>> We are on cassandra 1.2.2
>>>>
>>>> Thanks,
>>>> Dean
>>>>
>>>
>>>
>>>
>>>--
>>>Yuki Morishita
>>> t:yukim (http://twitter.com/yukim)
>>
>
>
>
>-- 
>Yuki Morishita
> t:yukim (http://twitter.com/yukim)


Mime
View raw message