systemml-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "LI Guobao (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SYSTEMML-2419) Setup and cleanup of remote workers
Date Fri, 13 Jul 2018 10:19:00 GMT

    [ https://issues.apache.org/jira/browse/SYSTEMML-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16542835#comment-16542835
] 

LI Guobao edited comment on SYSTEMML-2419 at 7/13/18 10:18 AM:
---------------------------------------------------------------

[~mboehm7], I have some questions. The first is about the setup of remote parfor worker.
In fact, I saw that this block of code is synchronized and so I wonder if it means that in
one executor, the parfor task will be launched in multi-threaded way? The second is, what is
the codegen class concretely is and used for? Because when parsing the parfor body, it will
generate a codegen class map.


was (Author: guobao):
[~mboehm7], I have some questions. The first is about the setup of remote parfor worker.
In fact, I saw that this block of code is synchronized and so I wonder if it means that in
one executor, the parfor task will be launched in multi-threaded way? The second is, what is
the codegen class used for? Because when parsing the parfor body, it will generate a codegen
class map.

> Setup and cleanup of remote workers
> -----------------------------------
>
>                 Key: SYSTEMML-2419
>                 URL: https://issues.apache.org/jira/browse/SYSTEMML-2419
>             Project: SystemML
>          Issue Type: Sub-task
>            Reporter: LI Guobao
>            Assignee: LI Guobao
>            Priority: Major
>
> In the context of distributed spark env, we need to firstly ship the necessary functions
and variables to the remote workers and then to initialize and register the cleanup of buffer
pool for each remote worker. All these are inspired by the parfor implementation.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message