cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jim Witschey (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-10298) Replaced dead node stayed in gossip forever
Date Mon, 14 Sep 2015 20:49:45 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-10298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14744236#comment-14744236
] 

Jim Witschey commented on CASSANDRA-10298:
------------------------------------------

[~dikanggu] Is this related to CASSANDRA-10321, and if so, how? You mentioned [in this comment|https://issues.apache.org/jira/browse/CASSANDRA-10321?focusedCommentId=14744120&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14744120]
that the node was running, but marked as having Thrift and Gossip unavailable. Is that the
same node you tried to remove? and could that unavailable state be the result of the failed
removal?

> Replaced dead node stayed in gossip forever
> -------------------------------------------
>
>                 Key: CASSANDRA-10298
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-10298
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Dikang Gu
>         Attachments: CASSANDRA-10298.patch
>
>
> The dead node stayed in the nodetool status,
> DN  10.210.165.55                    379.76 GB  256     ?       null
> And in the log, it throws NPE when trying to remove it.
> 2015-09-10_06:41:22.92453 ERROR 06:41:22 Exception in thread Thread[GossipStage:1,5,main]
> 2015-09-10_06:41:22.92454 java.lang.NullPointerException: null
> 2015-09-10_06:41:22.92455       at org.apache.cassandra.utils.UUIDGen.decompose(UUIDGen.java:100)

> 2015-09-10_06:41:22.92455       at org.apache.cassandra.db.HintedHandOffManager.deleteHintsForEndpoint(HintedHandOffManager.java:201)

> 2015-09-10_06:41:22.92455       at org.apache.cassandra.service.StorageService.excise(StorageService.java:1886)

> 2015-09-10_06:41:22.92455       at org.apache.cassandra.service.StorageService.excise(StorageService.java:1902)

> 2015-09-10_06:41:22.92456       at org.apache.cassandra.service.StorageService.handleStateLeft(StorageService.java:1805)
> 2015-09-10_06:41:22.92457       at org.apache.cassandra.service.StorageService.onChange(StorageService.java:1473)

> 2015-09-10_06:41:22.92457       at org.apache.cassandra.service.StorageService.onJoin(StorageService.java:2099)

> 2015-09-10_06:41:22.92457       at org.apache.cassandra.gms.Gossiper.handleMajorStateChange(Gossiper.java:1009)

> 2015-09-10_06:41:22.92458       at org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:1085)

> 2015-09-10_06:41:22.92458       at org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:49)

> 2015-09-10_06:41:22.92458       at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:62)

> 2015-09-10_06:41:22.92459       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
~[na:1.7.0_45]
> 2015-09-10_06:41:22.92460       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
~[na:1.7.0_45]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message