cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yabin Meng (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-10371) Decommissioned nodes can remain in gossip
Date Fri, 07 Oct 2016 18:56:22 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-10371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15555952#comment-15555952
] 

Yabin Meng commented on CASSANDRA-10371:
----------------------------------------

Hi,

I assume 2.2.8 should have this issue fixed. But in my CCM based 3-node cluster test, I still
see decommissioned node showing up in gossip. Below is what I did. Is there anything that
I miss here?

1) Bring up a CCM based 3 node cluster (version 2.2.8)
2) Decommission node3 (ccm node3 nodetool decommission)
3) On node1, run "nodetool describecluster" and got schema disagreement as below. Double checked
gossip info (ccm node1 nodetool gossipinfo) and still see node3 info.
Cluster Information:
	Name: c2.2.8
	Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch
	Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
	Schema versions:
		19d024c9-9762-35a0-931c-515c9d9d08a6: [127.0.0.1, 127.0.0.2]

		UNREACHABLE: [127.0.0.3]

> Decommissioned nodes can remain in gossip
> -----------------------------------------
>
>                 Key: CASSANDRA-10371
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-10371
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Distributed Metadata
>            Reporter: Brandon Williams
>            Assignee: Joel Knighton
>            Priority: Minor
>             Fix For: 2.1.14, 2.2.6, 3.0.4, 3.4
>
>
> This may apply to other dead states as well.  Dead states should be expired after 3 days.
 In the case of decom we attach a timestamp to let the other nodes know when it should be
expired.  It has been observed that sometimes a subset of nodes in the cluster never expire
the state, and through heap analysis of these nodes it is revealed that the epstate.isAlive
check returns true when it should return false, which would allow the state to be evicted.
 This may have been affected by CASSANDRA-8336.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message