From issues-return-8788-archive-asf-public=cust-asf.ponee.io@systemml.apache.org Sun Jun 24 21:00:05 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 47D23180662 for ; Sun, 24 Jun 2018 21:00:04 +0200 (CEST) Received: (qmail 7207 invoked by uid 500); 24 Jun 2018 19:00:03 -0000 Mailing-List: contact issues-help@systemml.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@systemml.apache.org Delivered-To: mailing list issues@systemml.apache.org Received: (qmail 7196 invoked by uid 99); 24 Jun 2018 19:00:03 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 24 Jun 2018 19:00:03 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 08BAB1A396E for ; Sun, 24 Jun 2018 19:00:03 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -110.301 X-Spam-Level: X-Spam-Status: No, score=-110.301 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id tIVTLcvqzEde for ; Sun, 24 Jun 2018 19:00:02 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id E1E745F4A0 for ; Sun, 24 Jun 2018 19:00:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 042D8E0FCF for ; Sun, 24 Jun 2018 19:00:01 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 5D3FF23FA0 for ; Sun, 24 Jun 2018 19:00:00 +0000 (UTC) Date: Sun, 24 Jun 2018 19:00:00 +0000 (UTC) From: "LI Guobao (JIRA)" To: issues@systemml.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (SYSTEMML-2420) Communication between ps and workers MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 LI Guobao created SYSTEMML-2420: ----------------------------------- Summary: Communication between ps and workers Key: SYSTEMML-2420 URL: https://issues.apache.org/jira/browse/SYSTEMML-2420 Project: SystemML Issue Type: Sub-task Reporter: LI Guobao Assignee: LI Guobao It aims to implement the parameter exchange between ps and workers. We could leverage spark RPC to setup a ps endpoint in driver node which means that the ps service could be discovered by workers in the network. And then the workers could invoke the pull/push method via RPC using the registered endpoint of ps service. Hence, in details, this tasks consists of registering the ps endpoint in spark rpc framework and using rpc to invoke target method in worker side. -- This message was sent by Atlassian JIRA (v7.6.3#76005)