cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dmitry Mitko (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-6565) New node refuses to join the ring.
Date Thu, 03 Jul 2014 07:39:24 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14051148#comment-14051148
] 

Dmitry Mitko edited comment on CASSANDRA-6565 at 7/3/14 7:38 AM:
-----------------------------------------------------------------

Sure, it is true. I've checked it then and have found, that source node has corrupted SSTable,
that can't be streamed successfully.

Right now, I've fixed that problem - to remove corrupted SSTable. I saw similar problems before
in the nodes' logs. Tried to fix it using cleanup and other nodetool modes - pity, but corrupted
SSTables wasn't fixed or repaired.

Can you advice some nice solution for them?

For example, if we have LeveledCompaction mode in the CF settings, how can we remove all of
SSTables, that belong to the same group?

P.S.: It will be so good, if failed to join node will write a lot of useful info into logs,
such as: "Probably, you have corrupted SSTable(s) in your source nodes, please, remove them
before joining".


was (Author: dv1983):
Sure, it is true. I've checked it then and have found, that source node has corrupted SSTable,
that can't be streamed successfully.

Right now, I've fixed that problem - to remove corrupted SSTable. I saw similar problems before
in the nodes' logs. Tried to fix it using cleanup and other nodetool modes - pity, but corrupted
SSTables wasn't fixed or repaired.

Can you advice some nice solution for them?

For example, if we have LeveledCompaction mode in the CF settings, how can we remove all of
SSTables, that belong to the same group?

> New node refuses to join the ring.
> ----------------------------------
>
>                 Key: CASSANDRA-6565
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-6565
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Shao-Chuan Wang
>
> We have 30 nodes in one DC, 25 nodes in another. We are running 2.0.1.
> Two nodes are joining the ring, but one of them failed
> ARN [STREAM-IN-/10.4.197.53] 2014-01-09 19:41:40,418 StreamResultFuture.java (line 209)
[Stream #e515d6e0-795d-11e3-b74a-b72892248056] Stream failed
> ERROR [main] 2014-01-09 19:41:40,418 CassandraDaemon.java (line 459) Exception encountered
during startup
> java.lang.RuntimeException: Error during boostrap: Stream failed
>         at org.apache.cassandra.dht.BootStrapper.bootstrap(BootStrapper.java:86)
>         at org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:901)
>         at org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:670)
>         at org.apache.cassandra.service.StorageService.initServer(StorageService.java:529)
>         at org.apache.cassandra.service.StorageService.initServer(StorageService.java:428)
>         at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:343)
>         at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:442)
>         at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:485)
> Caused by: org.apache.cassandra.streaming.StreamException: Stream failed
>         at org.apache.cassandra.streaming.StreamResultFuture.maybeComplete(StreamResultFuture.java:210)
>         at org.apache.cassandra.streaming.StreamResultFuture.handleSessionComplete(StreamResultFuture.java:185)
>         at org.apache.cassandra.streaming.StreamSession.closeSession(StreamSession.java:321)
>         at org.apache.cassandra.streaming.StreamSession.complete(StreamSession.java:501)
       at org.apache.cassandra.streaming.StreamSession.messageReceived(Stre
> amSession.java:376)        at org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:293)
>         at java.lang.Thread.run(Thread.java:744)
> ERROR [StorageServiceShutdownHook] 2014-01-09 19:41:40,428 CassandraDaemon.java (line
185) Exception in thread Thread[StorageServiceShutdownHook,5,main]
> java.lang.NullPointerException
>         at org.apache.cassandra.service.StorageService.stopRPCServer(StorageService.java:312)
>         at org.apache.cassandra.service.StorageService.shutdownClientServers(StorageService.java:361)
>         at org.apache.cassandra.service.StorageService.access$000(StorageService.java:96)
>         at org.apache.cassandra.service.StorageService$1.runMayThrow(StorageService.java:494)
>         at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
>         at java.lang.Thread.run(Thread.java:744)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message