cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Evans (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-9624) unable to bootstrap; streaming fails with NullPointerException
Date Wed, 29 Jul 2015 21:19:06 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-9624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14646776#comment-14646776
] 

Eric Evans commented on CASSANDRA-9624:
---------------------------------------

Since we had previously bootstrapped with 2.1.3 when the cluster had less data and lower throughput,
we scaled the cluster down to see if it would help, and in so doing, just managed to successfully
bootstrap a new node.

I don't even.

> unable to bootstrap; streaming fails with NullPointerException
> --------------------------------------------------------------
>
>                 Key: CASSANDRA-9624
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-9624
>             Project: Cassandra
>          Issue Type: Bug
>         Environment: Debian Jessie, 7u79-2.5.5-1~deb8u1, Cassandra 2.1.3
>            Reporter: Eric Evans
>            Assignee: Yuki Morishita
>             Fix For: 2.1.x
>
>
> When attempting to bootstrap a new node into a 2.1.3 cluster, the stream source fails
with a {{NullPointerException}}:
> {noformat}
> ERROR [STREAM-IN-/10.xx.x.xxx] 2015-06-13 00:02:01,264 StreamSession.java:477 - [Stream
#60e8c120-
> 115f-11e5-9fee-xxxxxxxxxxxx] Streaming error occurred
> java.lang.NullPointerException: null
>         at org.apache.cassandra.io.sstable.SSTableReader.getPositionsForRanges(SSTableReader.java:1277)
~[apache-cassandra-2.1.3.jar:2.1.3]
>         at org.apache.cassandra.streaming.StreamSession.getSSTableSectionsForRanges(StreamSession.java:313)
~[apache-cassandra-2.1.3.jar:2.1.3]
>         at org.apache.cassandra.streaming.StreamSession.addTransferRanges(StreamSession.java:266)
~[apache-cassandra-2.1.3.jar:2.1.3]
>         at org.apache.cassandra.streaming.StreamSession.prepare(StreamSession.java:493)
~[apache-cassandra-2.1.3.jar:2.1.3]
>         at org.apache.cassandra.streaming.StreamSession.messageReceived(StreamSession.java:425)
~[apache-cassandra-2.1.3.jar:2.1.3]
>         at org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:251)
~[apache-cassandra-2.1.3.jar:2.1.3]
>         at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]
> INFO  [STREAM-IN-/10.xx.x.xxx] 2015-06-13 00:02:01,265 StreamResultFuture.java:180 -
[Stream #60e8c120-115f-11e5-9fee-xxxxxxxxxxxx] Session with /10.xx.x.xx1 is complete
> {noformat}
> _Update (2015-06-26):_
> I can also reproduce this on 2.1.7, though without the NPE on the stream-from side.
> Stream source / existing node:
> {noformat}
> INFO  [STREAM-IN-/10.64.32.178] 2015-06-26 06:48:53,060 StreamResultFuture.java:180 -
[Stream #8bdeb1b0-1ad2-11e5-abd8-3fcfb96209d9] Session with /10.64.32.178 is complete
> INFO  [STREAM-IN-/10.64.32.178] 2015-06-26 06:48:53,064 StreamResultFuture.java:212 -
[Stream #8bdeb1b0-1ad2-11e5-abd8-3fcfb96209d9] All sessions completed
> {noformat}
> Stream sink / bootstrapping node:
> {noformat}
> INFO  [StreamReceiveTask:57] 2015-06-26 06:48:53,061 StreamResultFuture.java:180 - [Stream
#8bdeb1b0-1ad2-11e5-abd8-3fcfb96209d9] Session with /10.64.32.160 is complete
> WARN  [StreamReceiveTask:57] 2015-06-26 06:48:53,062 StreamResultFuture.java:207 - [Stream
#8bdeb1b0-1ad2-11e5-abd8-3fcfb96209d9] Stream failed
> INFO  [CompactionExecutor:2885] 2015-06-26 06:48:53,062 ColumnFamilyStore.java:906 -
Enqueuing flush of compactions_in_progress: 428 (0%) on-heap, 379 (0%) off-heap
> INFO  [MemtableFlushWriter:959] 2015-06-26 06:48:53,063 Memtable.java:346 - Writing Memtable-compactions_in_progress@1203013482(294
serialized bytes, 12 ops, 0%/0% of on/off-heap limit)
> ERROR [main] 2015-06-26 06:48:53,063 CassandraDaemon.java:541 - Exception encountered
during startup
> java.lang.RuntimeException: Error during boostrap: Stream failed
>         at org.apache.cassandra.dht.BootStrapper.bootstrap(BootStrapper.java:86) ~[apache-cassandra-2.1.7.jar:2.1.7]
>         at org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:1137)
~[apache-cassandra-2.1.7.jar:2.1.7]
>         at org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:927)
~[apache-cassandra-2.1.7.jar:2.1.7]
>         at org.apache.cassandra.service.StorageService.initServer(StorageService.java:723)
~[apache-cassandra-2.1.7.jar:2.1.7]
>         at org.apache.cassandra.service.StorageService.initServer(StorageService.java:605)
~[apache-cassandra-2.1.7.jar:2.1.7]
>         at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:378)
[apache-cassandra-2.1.7.jar:2.1.7]
>         at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:524)
[apache-cassandra-2.1.7.jar:2.1.7]
>         at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:613)
[apache-cassandra-2.1.7.jar:2.1.7]
> Caused by: org.apache.cassandra.streaming.StreamException: Stream failed
>         at org.apache.cassandra.streaming.management.StreamEventJMXNotifier.onFailure(StreamEventJMXNotifier.java:85)
~[apache-cassandra-2.1.7.jar:2.1.7]
>         at com.google.common.util.concurrent.Futures$4.run(Futures.java:1172) ~[guava-16.0.jar:na]
>         at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)
~[guava-16.0.jar:na]
>         at com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156)
~[guava-16.0.jar:na]
>         at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:145)
~[guava-16.0.jar:na]
>         at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:202)
~[guava-16.0.jar:na]
>         at org.apache.cassandra.streaming.StreamResultFuture.maybeComplete(StreamResultFuture.java:208)
~[apache-cassandra-2.1.7.jar:2.1.7]
>         at org.apache.cassandra.streaming.StreamResultFuture.handleSessionComplete(StreamResultFuture.java:184)
~[apache-cassandra-2.1.7.jar:2.1.7]
>         at org.apache.cassandra.streaming.StreamSession.closeSession(StreamSession.java:409)
~[apache-cassandra-2.1.7.jar:2.1.7]
>         at org.apache.cassandra.streaming.StreamSession.maybeCompleted(StreamSession.java:682)
~[apache-cassandra-2.1.7.jar:2.1.7]
>         at org.apache.cassandra.streaming.StreamSession.taskCompleted(StreamSession.java:645)
~[apache-cassandra-2.1.7.jar:2.1.7]
>         at org.apache.cassandra.streaming.StreamReceiveTask$OnCompletionRunnable.run(StreamReceiveTask.java:139)
~[apache-cassandra-2.1.7.jar:2.1.7]
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) ~[na:1.7.0_79]
>         at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_79]
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
~[na:1.7.0_79]
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
~[na:1.7.0_79]
>         at java.lang.Thread.run(Thread.java:745) ~[na:1.7.0_79]
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message