cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brandon Williams (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-4347) IP change of node requires assassinate to really remove old IP
Date Tue, 26 Jun 2012 14:53:44 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-4347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13401441#comment-13401441
] 

Brandon Williams commented on CASSANDRA-4347:
---------------------------------------------

Do you have logs from cass03 around the time of the move?  I ask because ALL nodes should
have logged the "Nodes [new] and [old] have the same token xxxx.  [new] is the new owner"
message once, as well as the fat client removal message.  If the messages are repeating though,
that's a sign that something is wrong.

I ask for cass03 because it still shows 10.12.8.97 as the owner of the token for some reason,
which would keep trying to propagate it to the rest of the cluster and cause the messages.


                
> IP change of node requires assassinate to really remove old IP
> --------------------------------------------------------------
>
>                 Key: CASSANDRA-4347
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4347
>             Project: Cassandra
>          Issue Type: Bug
>    Affects Versions: 1.0.10
>         Environment: RHEL6, 64bit
>            Reporter: Karl Mueller
>            Assignee: Brandon Williams
>            Priority: Minor
>         Attachments: LocationInfo-hd-279-Data.db, dev-cass-post-assassinate-gossipinfo.txt,
kaos-cass00-gossipinfo-postmove.txt, kaos-cass03-gossipinfo-postmove.txt
>
>
> In changing the IP addresses of nodes one-by-one, the node successfully moves itself
and its token.  Everything works properly.
> However, the node which had its IP changed (but NOT other nodes in the ring) continues
to have some type of state associated with the old IP and produces log messages like this:
>  INFO [GossipStage:1] 2012-06-15 15:25:01,490 Gossiper.java (line 838) Node /10.12.9.157
is now part of the cluster
>  INFO [GossipStage:1] 2012-06-15 15:25:01,490 Gossiper.java (line 804) InetAddress /10.12.9.157
is now UP
>  INFO [GossipStage:1] 2012-06-15 15:25:01,491 StorageService.java (line 1017) Nodes /10.12.9.157
and dev-cass01.sv.walmartlabs.com/10.93.15.11 have the same token 113427455640312821154458202477256070484.
 Ignoring /10.12.9.157
>  INFO [GossipTasks:1] 2012-06-15 15:25:11,373 Gossiper.java (line 818) InetAddress /10.12.9.157
is now dead.
>  INFO [GossipTasks:1] 2012-06-15 15:25:32,380 Gossiper.java (line 632) FatClient /10.12.9.157
has been silent for 30000ms, removing from gossip
>  INFO [GossipStage:1] 2012-06-15 15:26:32,490 Gossiper.java (line 838) Node /10.12.9.157
is now part of the cluster
>  INFO [GossipStage:1] 2012-06-15 15:26:32,491 Gossiper.java (line 804) InetAddress /10.12.9.157
is now UP
>  INFO [GossipStage:1] 2012-06-15 15:26:32,491 StorageService.java (line 1017) Nodes /10.12.9.157
and dev-cass01.sv.walmartlabs.com/10.93.15.11 have the same token 113427455640312821154458202477256070484.
 Ignoring /10.12.9.157
>  INFO [GossipTasks:1] 2012-06-15 15:26:42,402 Gossiper.java (line 818) InetAddress /10.12.9.157
is now dead.
>  INFO [GossipTasks:1] 2012-06-15 15:27:03,410 Gossiper.java (line 632) FatClient /10.12.9.157
has been silent for 30000ms, removing from gossip
>  INFO [GossipStage:1] 2012-06-15 15:28:04,533 Gossiper.java (line 838) Node /10.12.9.157
is now part of the cluster
> Other nodes do NOT have the old IP showing up in logs.  It's only the node that moved.
> The old IP doesn't show up in ring anywhere or in any other fashion.  The cluster seems
to be fully operational, so I think it's just a cleanup issue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message