cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Paulo Motta (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-10111) reconnecting snitch can bypass cluster name check
Date Mon, 21 Dec 2015 21:00:47 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-10111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067055#comment-15067055
] 

Paulo Motta commented on CASSANDRA-10111:
-----------------------------------------

Code and approach looks good but it seems we can only bump the messaging service version in
the next major, or 4.0.

Alternatives are to wait until then, or maybe do some workaround before, like adding a new
gossip field {{CLUSTER_ID}} to {{ApplicationState}} and ignore any {{GossipDigestAck}} or
{{GossipDigestAck2}} messages containing states from other cluster ids.

Since this is quite an unlikely (and unfortunate) situation I'd be more in favor of waiting
for the messaging bump (since we've waited until here) instead of polluting gossip with more
fields.

> reconnecting snitch can bypass cluster name check
> -------------------------------------------------
>
>                 Key: CASSANDRA-10111
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-10111
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Distributed Metadata
>            Reporter: Chris Burroughs
>            Assignee: Joel Knighton
>              Labels: gossip
>             Fix For: 3.x
>
>
> Setup:
>  * Two clusters: A & B
>  * Both are two DC cluster
>  * Both use GossipingPropertyFileSnitch with different listen_address/broadcast_address
> A new node was added to cluster A with a broadcast_address of an existing node in cluster
B (due to an out of data DNS entry).  Cluster B  added all of the nodes from cluster A, somehow
bypassing the cluster name mismatch check for this nodes.  The first reference to cluster
A nodes in cluster B logs is when then were added:
> {noformat}
>  INFO [GossipStage:1] 2015-08-17 15:08:33,858 Gossiper.java (line 983) Node /8.37.70.168
is now part of the cluster
> {noformat}
> Cluster B nodes then tried to gossip to cluster A nodes, but cluster A kept them out
with 'ClusterName mismatch'.  Cluster B however tried to send to send reads/writes to cluster
A and general mayhem ensued.
> Obviously this is a Bad (TM) config that Should Not Be Done.  However, since the consequence
of crazy merged clusters are really bad (the reason there is the name mismatch check in the
first place) I think the hole is reasonable to plug.  I'm not sure exactly what the code path
is that skips the check in GossipDigestSynVerbHandler.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message