Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Date: Tue, 5 Jul 2016 15:32:11 +0000 (UTC)
From: "Jeremiah Jordan (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: <JIRA.12966430.1462831232000.26533.1467732731292@Atlassian.JIRA>
In-Reply-To: <JIRA.12966430.1462831232000@Atlassian.JIRA>
References: <JIRA.12966430.1462831232000@Atlassian.JIRA> <JIRA.12966430.1462831232460@arcas>
Subject: [jira] [Commented] (CASSANDRA-11740) Nodes have wrong membership
 view of the cluster
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
archived-at: Tue, 05 Jul 2016 15:32:13 -0000


    [ https://issues.apache.org/jira/browse/CASSANDRA-11740?page=3Dcom.atla=
ssian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=
=3D15362641#comment-15362641 ]=20

Jeremiah Jordan commented on CASSANDRA-11740:
---------------------------------------------

[~dikanggu] Yes, the peers table should be the truth.

[~jkni] we should probably switch the order of those checks.  I would think=
 we should check the peers table before checking the cassandra-topology.pro=
perties.  The peers table will contain the information from the last time w=
e got gossip status from those nodes, so if there are values in it we shoul=
d probably use those before seeing if the cassandra-topology.properties has=
 something.

> Nodes have wrong membership view of the cluster
> -----------------------------------------------
>
>                 Key: CASSANDRA-11740
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-1174=
0
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Dikang Gu
>            Assignee: Joel Knighton
>             Fix For: 2.2.x, 3.x
>
>
> We have a few hundreds nodes across 3 data centers, and we are doing a fe=
w millions writes per second into the cluster.
> The problem we found is that there are some nodes (>10) have very wrong v=
iew of the cluster.
> For example, we have 3 data centers A, B and C. On the problem nodes, in =
the output of the 'nodetool status', it shows that ~100 nodes are not in da=
ta center A, B, or C. Instead, it shows nodes are in DC1, and rack r1, whic=
h is very wrong. And as a result, the node will return wrong results to cli=
ent requests.
> {code}
> Datacenter: DC1
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> Status=3DUp/Down
> / State=3DNormal/Leaving/Joining/Moving
> =E2=80=93 Address Load Tokens Owns Host ID Rack
> UN 2401:db00:11:6134:face:0:1:0 509.52 GB 256 ? e24656ac-c3b2-4117-b933-a=
5b06852c993 r1
> UN 2401:db00:11:b218:face:0:5:0 510.01 GB 256 ? 53da2104-b1b5-4fa5-a3dd-5=
2c7557149f9 r1
> UN 2401:db00:2130:5133:face:0:4d:0 459.75 GB 256 ? ef8311f0-f6b8-491c-904=
d-baa925cdd7c2 r1
> {code}
> We are using GossipingPropertyFileSnitch.
> Thanks


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)