Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Date: Wed, 22 Feb 2017 14:37:44 +0000 (UTC)
From: "Jason Brown (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: <JIRA.13005368.1474014789000.31013.1487774264670@Atlassian.JIRA>
In-Reply-To: <JIRA.13005368.1474014789000@Atlassian.JIRA>
References: <JIRA.13005368.1474014789000@Atlassian.JIRA> <JIRA.13005368.1474014789915@jira-lw-us.apache.org>
Subject: [jira] [Commented] (CASSANDRA-12653) In-flight shadow round
 requests
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Wed, 22 Feb 2017 14:37:50 -0000


    [ https://issues.apache.org/jira/browse/CASSANDRA-12653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15878328#comment-15878328 ] 

Jason Brown commented on CASSANDRA-12653:
-----------------------------------------

On the whole, I'm pretty good with this patch - nice work, Stefan!

I have two nits:
- the 3.11 and trunk branches have the tests included, but 2.2 and 3.0 do not. I that just an oversight? Can we add the tests to those branches, as well?
- {{GossipDigestAckVerbHandler#doVerb}}, you get the [following timestamp|https://github.com/spodkowinski/cassandra/commit/9179b7ee06c51a79881f5be18cd01261ebe62143#diff-787d4963a51f20221468e976df1b121aR64]: {code} long ts = epStateMap.values().iterator().next().getUpdateTimestamp();{code}
We [do not serialize|https://github.com/apache/cassandra/blob/cassandra-2.2/src/java/org/apache/cassandra/gms/EndpointState.java#L166] {{EndpointState.updateTimestamp}}, so the value at the receiver ends up being the receiver's {{System.nanoTime()}}, as can be seen from the [{{EndpointState}} constructor|https://github.com/apache/cassandra/blob/cassandra-2.2/src/java/org/apache/cassandra/gms/EndpointState.java#L58]. What is it you are looking to confirm here? I'm kinda leaning toward eliminating that check altogether as {code}Gossiper.instance.firstSynSendAt == 0{code} might be a sufficient check anyways.


> In-flight shadow round requests
> -------------------------------
>
>                 Key: CASSANDRA-12653
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-12653
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Distributed Metadata
>            Reporter: Stefan Podkowinski
>            Assignee: Stefan Podkowinski
>            Priority: Minor
>             Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x
>
>
> Bootstrapping or replacing a node in the cluster requires to gather and check some host IDs or tokens by doing a gossip "shadow round" once before joining the cluster. This is done by sending a gossip SYN to all seeds until we receive a response with the cluster state, from where we can move on in the bootstrap process. Receiving a response will call the shadow round done and calls {{Gossiper.resetEndpointStateMap}} for cleaning up the received state again.
> The issue here is that at this point there might be other in-flight requests and it's very likely that shadow round responses from other seeds will be received afterwards, while the current state of the bootstrap process doesn't expect this to happen (e.g. gossiper may or may not be enabled). 
> One side effect will be that MigrationTasks are spawned for each shadow round reply except the first. Tasks might or might not execute based on whether at execution time {{Gossiper.resetEndpointStateMap}} had been called, which effects the outcome of {{FailureDetector.instance.isAlive(endpoint))}} at start of the task. You'll see error log messages such as follows when this happend:
> {noformat}
> INFO  [SharedPool-Worker-1] 2016-09-08 08:36:39,255 Gossiper.java:993 - InetAddress /xx.xx.xx.xx is now UP
> ERROR [MigrationStage:1]    2016-09-08 08:36:39,255 FailureDetector.java:223 - unknown endpoint /xx.xx.xx.xx
> {noformat}
> Although is isn't pretty, I currently don't see any serious harm from this, but it would be good to get a second opinion (feel free to close as "wont fix").
> /cc [~Stefania] [~thobbs]


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)