Mailing-List: contact dev-help@reef.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@reef.apache.org
Date: Mon, 5 Jun 2017 20:39:04 +0000 (UTC)
From: "Doug Service (JIRA)" <jira@apache.org>
To: dev@reef.apache.org
Message-ID: <JIRA.13060772.1490988247000.368629.1496695144443@Atlassian.JIRA>
In-Reply-To: <JIRA.13060772.1490988247000@Atlassian.JIRA>
References: <JIRA.13060772.1490988247000@Atlassian.JIRA> <JIRA.13060772.1490988247804@jira-lw-us.apache.org>
Subject: [jira] [Comment Edited] (REEF-1763) Replace interop call to
 StartHandlerOnNext() between Java/C# bridges
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Mon, 05 Jun 2017 20:42:59 -0000


    [ https://issues.apache.org/jira/browse/REEF-1763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16037538#comment-16037538 ] 

Doug Service edited comment on REEF-1763 at 6/5/17 8:38 PM:
------------------------------------------------------------

Answer to comment one: 

Yes the goal is to replace all of the interop calls with Avro messages. There was a consensus at the time this was proposed that by providing a well defined Avro interface it would allow us to support bridges in a number of different languages.

I am also concerned about the performance impact of sending the messages over the network. The existing bridge design seems to rely on the synchronous nature of the interop calls. If the interface was asynchronous or at least partially asynchronous, the throughput could still be high even though any one given message takes longer to complete since multiple messages can be in flight at the same time.

The current pull request specifically addresses the fact that lot of messages will be needed to complete the work by providing a framework where all messages in org.apache.reef.bridge.message package are  discovered at system startup using refection on both the C# and Java sides and then serializers and deserializers are programatically constructed. This means that replacing a given interop call requires defining the associated messages and implementing the calls on the java and C# sides as the bridge network will automatically build what is needed to send and receive the new message. 


was (Author: dougservice):
Answer to comment one: 

Yes the goal is to replace all of the interop calls with Avro messages. There was a consensus at the time this was proposed that by providing a well defined Avro interface it would allow us to support bridges in a number of different languages.

I am also concerned about the performance impact of sending the messages over the network. The existing bridge design seems to rely on the synchronous nature of the interop calls. If the interface was asynchronous or at least partially asynchronous, the throughput could still be high even though any one given message takes longer to complete since multiple messages can be in flight at the same time.

The current pull request specifically addresses the fact that lot of messages will be needed to complete the work by providing a framework where all messages in org.apache.reef.bridge.message are  discovered at system startup using refection on both the C# and Java sides and then serializers and deserializers are programatically constructed. This means that replacing a given interop call requires defining the associated messages and implementing the calls on the java and C# sides as the bridge network will automatically build what is needed to send and receive the new message. 

> Replace interop call to StartHandlerOnNext() between Java/C# bridges
> --------------------------------------------------------------------
>
>                 Key: REEF-1763
>                 URL: https://issues.apache.org/jira/browse/REEF-1763
>             Project: REEF
>          Issue Type: Sub-task
>          Components: REEF Bridge, REEF Driver
>            Reporter: Doug Service
>            Assignee: Doug Service
>
> Currently bridge communication from Java to C# is performed via managed C++ interop which is not supported on Linux. This is the first of a number of changes to replace all of the interop calls between Java and C# with Avro messages carried by the REEF Wake networking classes.


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)