cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dikang Gu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-10298) Replaced dead node stayed in gossip forever
Date Fri, 11 Sep 2015 06:23:46 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-10298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14740279#comment-14740279
] 

Dikang Gu commented on CASSANDRA-10298:
---------------------------------------

The new node successfully finished the replacement, but for some reason, the dead node still
stayed in the nodetool status for some other live nodes.

I can not remove it from the gossip, even after restarting some live node.

> Replaced dead node stayed in gossip forever
> -------------------------------------------
>
>                 Key: CASSANDRA-10298
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-10298
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Dikang Gu
>         Attachments: CASSANDRA-10298.patch
>
>
> The dead node stayed in the nodetool status,
> DN  10.210.165.55                    379.76 GB  256     ?       null
> And in the log, it throws NPE when trying to remove it.
> 2015-09-10_06:41:22.92453 ERROR 06:41:22 Exception in thread Thread[GossipStage:1,5,main]
> 2015-09-10_06:41:22.92454 java.lang.NullPointerException: null
> 2015-09-10_06:41:22.92455       at org.apache.cassandra.utils.UUIDGen.decompose(UUIDGen.java:100)

> 2015-09-10_06:41:22.92455       at org.apache.cassandra.db.HintedHandOffManager.deleteHintsForEndpoint(HintedHandOffManager.java:201)

> 2015-09-10_06:41:22.92455       at org.apache.cassandra.service.StorageService.excise(StorageService.java:1886)

> 2015-09-10_06:41:22.92455       at org.apache.cassandra.service.StorageService.excise(StorageService.java:1902)

> 2015-09-10_06:41:22.92456       at org.apache.cassandra.service.StorageService.handleStateLeft(StorageService.java:1805)
> 2015-09-10_06:41:22.92457       at org.apache.cassandra.service.StorageService.onChange(StorageService.java:1473)

> 2015-09-10_06:41:22.92457       at org.apache.cassandra.service.StorageService.onJoin(StorageService.java:2099)

> 2015-09-10_06:41:22.92457       at org.apache.cassandra.gms.Gossiper.handleMajorStateChange(Gossiper.java:1009)

> 2015-09-10_06:41:22.92458       at org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:1085)

> 2015-09-10_06:41:22.92458       at org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:49)

> 2015-09-10_06:41:22.92458       at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:62)

> 2015-09-10_06:41:22.92459       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
~[na:1.7.0_45]
> 2015-09-10_06:41:22.92460       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
~[na:1.7.0_45]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message