Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Date: Fri, 7 Oct 2016 18:56:22 +0000 (UTC)
From: "Yabin Meng (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: <JIRA.12875742.1442536936000.766387.1475866582125@Atlassian.JIRA>
In-Reply-To: <JIRA.12875742.1442536936000@Atlassian.JIRA>
References: <JIRA.12875742.1442536936000@Atlassian.JIRA> <JIRA.12875742.1442536936696@arcas>
Subject: [jira] [Commented] (CASSANDRA-10371) Decommissioned nodes can
 remain in gossip
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Fri, 07 Oct 2016 18:56:24 -0000


    [ https://issues.apache.org/jira/browse/CASSANDRA-10371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15555952#comment-15555952 ] 

Yabin Meng commented on CASSANDRA-10371:
----------------------------------------

Hi,

I assume 2.2.8 should have this issue fixed. But in my CCM based 3-node cluster test, I still see decommissioned node showing up in gossip. Below is what I did. Is there anything that I miss here?

1) Bring up a CCM based 3 node cluster (version 2.2.8)
2) Decommission node3 (ccm node3 nodetool decommission)
3) On node1, run "nodetool describecluster" and got schema disagreement as below. Double checked gossip info (ccm node1 nodetool gossipinfo) and still see node3 info.
Cluster Information:
	Name: c2.2.8
	Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch
	Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
	Schema versions:
		19d024c9-9762-35a0-931c-515c9d9d08a6: [127.0.0.1, 127.0.0.2]

		UNREACHABLE: [127.0.0.3]

> Decommissioned nodes can remain in gossip
> -----------------------------------------
>
>                 Key: CASSANDRA-10371
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-10371
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Distributed Metadata
>            Reporter: Brandon Williams
>            Assignee: Joel Knighton
>            Priority: Minor
>             Fix For: 2.1.14, 2.2.6, 3.0.4, 3.4
>
>
> This may apply to other dead states as well.  Dead states should be expired after 3 days.  In the case of decom we attach a timestamp to let the other nodes know when it should be expired.  It has been observed that sometimes a subset of nodes in the cluster never expire the state, and through heap analysis of these nodes it is revealed that the epstate.isAlive check returns true when it should return false, which would allow the state to be evicted.  This may have been affected by CASSANDRA-8336.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)