systemml-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "LI Guobao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SYSTEMML-2420) Communication between ps and workers
Date Sun, 24 Jun 2018 22:15:00 GMT

    [ https://issues.apache.org/jira/browse/SYSTEMML-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16521704#comment-16521704
] 

LI Guobao commented on SYSTEMML-2420:
-------------------------------------

[~mboehm7], I wrote down the idea for implementing the exchange between ps and workers. And
is this idea correct?

> Communication between ps and workers
> ------------------------------------
>
>                 Key: SYSTEMML-2420
>                 URL: https://issues.apache.org/jira/browse/SYSTEMML-2420
>             Project: SystemML
>          Issue Type: Sub-task
>            Reporter: LI Guobao
>            Assignee: LI Guobao
>            Priority: Major
>
> It aims to implement the parameter exchange between ps and workers. We could leverage
spark RPC to setup a ps endpoint in driver node which means that the ps service could be discovered
by workers in the network. And then the workers could invoke the pull/push method via RPC
using the registered endpoint of ps service. Hence, in details, this tasks consists of registering
the ps endpoint in spark rpc framework and using rpc to invoke target method in worker side.
We can learn that the spark rpc is implemented in Scala. Hence we need to wrap them in in
order to be used in Java. Overall, we could register the ps service with _RpcEndpoint_ and
invoke the service with _RpcEndpointRef_.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message