systemml-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "LI Guobao (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SYSTEMML-2420) Communication between ps and workers
Date Sun, 24 Jun 2018 21:45:00 GMT

     [ https://issues.apache.org/jira/browse/SYSTEMML-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

LI Guobao updated SYSTEMML-2420:
--------------------------------
    Description: It aims to implement the parameter exchange between ps and workers. We could
leverage spark RPC to setup a ps endpoint in driver node which means that the ps service could
be discovered by workers in the network. And then the workers could invoke the pull/push method
via RPC using the registered endpoint of ps service. Hence, in details, this tasks consists
of registering the ps endpoint in spark rpc framework and using rpc to invoke target method
in worker side. We can learn that the spark rpc is implemented in Scala. But we could easily
wrap them in java class to be reused.  (was: It aims to implement the parameter exchange between
ps and workers. We could leverage spark RPC to setup a ps endpoint in driver node which means
that the ps service could be discovered by workers in the network. And then the workers could
invoke the pull/push method via RPC using the registered endpoint of ps service. Hence, in
details, this tasks consists of registering the ps endpoint in spark rpc framework and using
rpc to invoke target method in worker side.)

> Communication between ps and workers
> ------------------------------------
>
>                 Key: SYSTEMML-2420
>                 URL: https://issues.apache.org/jira/browse/SYSTEMML-2420
>             Project: SystemML
>          Issue Type: Sub-task
>            Reporter: LI Guobao
>            Assignee: LI Guobao
>            Priority: Major
>
> It aims to implement the parameter exchange between ps and workers. We could leverage
spark RPC to setup a ps endpoint in driver node which means that the ps service could be discovered
by workers in the network. And then the workers could invoke the pull/push method via RPC
using the registered endpoint of ps service. Hence, in details, this tasks consists of registering
the ps endpoint in spark rpc framework and using rpc to invoke target method in worker side.
We can learn that the spark rpc is implemented in Scala. But we could easily wrap them in
java class to be reused.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message