cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Stevens (JIRA)" <>
Subject [jira] [Created] (CASSANDRA-8801) Decommissioned nodes are willing to rejoin the cluster if restarted
Date Fri, 13 Feb 2015 15:04:11 GMT
Eric Stevens created CASSANDRA-8801:

             Summary: Decommissioned nodes are willing to rejoin the cluster if restarted
                 Key: CASSANDRA-8801
             Project: Cassandra
          Issue Type: Bug
          Components: Core
            Reporter: Eric Stevens
            Priority: Critical

This issue comes from the Cassandra user group.

If a node which was successfully decommissioned gets restarted with its data directory in
tact, it will rejoin the cluster immediately going to {{UN}} and beginning to serve client

This is wrong - the node has consistency issues, having missed any writes while it was offline
because no hinted handoffs were being kept.  And in the best case scenario (it's spotted and
remediated immediately), near-100% overstreaming will still occur.

Also, whatever reasons the operator had for decommissioning the node would presumably still
be valid, so this action may threaten cluster stability if the node is underpowered or suffering
hardware issues.

But what elevates this to critical is that if the node had been offline longer than gc_grace_seconds,
it may cause permanent and unrecoverable consistency issues due to data resurrection.

h3. Recommendation:
A node should remember that it was decommissioned and refuse to rejoin a cluster without at
least a -Dflag forcing it to.  

This message was sent by Atlassian JIRA

View raw message