cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Ellis (JIRA)" <j...@apache.org>
Subject [jira] Issue Comment Edited: (CASSANDRA-1609) Cluster restart re-adds removed tokens
Date Tue, 12 Oct 2010 21:56:37 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12920368#action_12920368
] 

Jonathan Ellis edited comment on CASSANDRA-1609 at 10/12/10 5:55 PM:
---------------------------------------------------------------------

ring state management is a mess.  token removal happens in 3 places:

 1) node receives decommission notice (STATE_LEFT)
 2) node receives removetoken notice, piggy-backed on STATE_NORMAL
 3) node coordinates removetoken (gossiper will not trigger notifications for state changes
that initiated locally, so this needs to be handled separately from 2)

2) was updating the SystemTable w/ the removal but the others were not.

patch attached to move this logic into excise() method and call from all 3 places.

      was (Author: jbellis):
    ring state management is a mess.  token removal happens in 3 places:

 1) node receives decommission notice (STATE_LEFT)
 2) node receives removetoken notice, piggy-backed on STATE_NORMAL
 3) node coordinates removetoken (gossiper will not trigger notifications for state changes
that initiated locally, so this needs to be handled separately from 2)

2) was happening but the others were not.

patch attached to move this logic into excise() method and call from all 3 places.
  
> Cluster restart re-adds removed tokens
> --------------------------------------
>
>                 Key: CASSANDRA-1609
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-1609
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.7 beta 2
>            Reporter: Nick Bailey
>            Assignee: Jonathan Ellis
>             Fix For: 0.7.0
>
>         Attachments: 1609.txt
>
>
> After a cluster restart one of our nodes began reporting tokens that had been removed
a good while ago (week or more) in it's nodetool ring output.  This probably has something
to do with our change to persist the ring in CASSANDRA-1518 and removetoken changes in CASSANDRA-1216.
The node didn't actually gossip the removed tokens so they showed up in TMD but not gossip.
> Additionally all nodes began reporting a node that had been removed maybe an hour ago.
 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message