cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeremiah Jordan (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-11740) Nodes have wrong membership view of the cluster
Date Tue, 05 Jul 2016 15:32:11 GMT


Jeremiah Jordan commented on CASSANDRA-11740:

[~dikanggu] Yes, the peers table should be the truth.

[~jkni] we should probably switch the order of those checks.  I would think we should check
the peers table before checking the  The peers table will contain
the information from the last time we got gossip status from those nodes, so if there are
values in it we should probably use those before seeing if the
has something.

> Nodes have wrong membership view of the cluster
> -----------------------------------------------
>                 Key: CASSANDRA-11740
>                 URL:
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Dikang Gu
>            Assignee: Joel Knighton
>             Fix For: 2.2.x, 3.x
> We have a few hundreds nodes across 3 data centers, and we are doing a few millions writes
per second into the cluster.
> The problem we found is that there are some nodes (>10) have very wrong view of the
> For example, we have 3 data centers A, B and C. On the problem nodes, in the output of
the 'nodetool status', it shows that ~100 nodes are not in data center A, B, or C. Instead,
it shows nodes are in DC1, and rack r1, which is very wrong. And as a result, the node will
return wrong results to client requests.
> {code}
> Datacenter: DC1
> ===============
> Status=Up/Down
> / State=Normal/Leaving/Joining/Moving
> – Address Load Tokens Owns Host ID Rack
> UN 2401:db00:11:6134:face:0:1:0 509.52 GB 256 ? e24656ac-c3b2-4117-b933-a5b06852c993
> UN 2401:db00:11:b218:face:0:5:0 510.01 GB 256 ? 53da2104-b1b5-4fa5-a3dd-52c7557149f9
> UN 2401:db00:2130:5133:face:0:4d:0 459.75 GB 256 ? ef8311f0-f6b8-491c-904d-baa925cdd7c2
> {code}
> We are using GossipingPropertyFileSnitch.
> Thanks

This message was sent by Atlassian JIRA

View raw message