ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexandr Kuramshin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (IGNITE-4111) Communication fails to send message if target node did not finish join process
Date Sun, 20 Nov 2016 23:53:59 GMT

    [ https://issues.apache.org/jira/browse/IGNITE-4111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15682058#comment-15682058
] 

Alexandr Kuramshin commented on IGNITE-4111:
--------------------------------------------

I've found that the actual lock occurs in the TcpCommunicationSpi.srvLsnr.onConnected() method
while sending the node ID on accepting a remote connection:

ses.send(nodeIdMessage());

This invocation causes awaiting the ctxInitLatch, hence locks a clientThread of the GridNioServer
until the node in not joined the topology.

At the same time invocation of the ch.read(buf) is being blocked and this call will never
be reached:

UUID rmtNodeId0 = U.bytesToUuid(buf.array(), 1);

In consequence of timeout.

> Communication fails to send message if target node did not finish join process
> ------------------------------------------------------------------------------
>
>                 Key: IGNITE-4111
>                 URL: https://issues.apache.org/jira/browse/IGNITE-4111
>             Project: Ignite
>          Issue Type: Bug
>          Components: general
>            Reporter: Semen Boikov
>            Assignee: Alexandr Kuramshin
>             Fix For: 2.0
>
>         Attachments: test onFirstMessage hang.log
>
>
> Currently this scenario is possible:
> - joining node sent join request and waits for TcpDiscoveryNodeAddFinishedMessage inside
ServerImpl.joinTopology
> - others nodes already see this node and can send messages to it (for example try to
run compute job on this node)
> - joining node can not receive message: TcpCommunicationSpi will hang inside 'onFirstMessage'
on 'getSpiContext' call, so sending node will get error trying to establish connection
> Possible fix: if in onFirstMessage() spi context is not available, then TcpCommunicationSpi
 should send special response which indicates that this node is not ready yet, and sender
should retry after some time.
> Also need check internal code for places where message can be unnecessarily sent to node:
one such place is GridCachePartitionExchangeManager.refreshPartitions - message is sent to
all known nodes, but here we can filter by node order / finished exchage version.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message