systemml-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arvind Surve (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SYSTEMML-1830) Improve the data locality for the tasks in ParFor body
Date Tue, 29 Aug 2017 22:20:00 GMT

     [ https://issues.apache.org/jira/browse/SYSTEMML-1830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Arvind Surve updated SYSTEMML-1830:
-----------------------------------
    Sprint: Sprint 5  (was: Sprint 4)

> Improve the data locality for the tasks in ParFor body
> ------------------------------------------------------
>
>                 Key: SYSTEMML-1830
>                 URL: https://issues.apache.org/jira/browse/SYSTEMML-1830
>             Project: SystemML
>          Issue Type: Improvement
>    Affects Versions: SystemML 1.0
>            Reporter: Fei Hu
>            Assignee: Fei Hu
>              Labels: ParFor
>
> For {{RemoteParForSpark}}, the tasks are parallelized without considering the data locality
of the input matrixes. It will cause a lot of data shuffling if the volume of the input data
size is large. 
> We can predict the data location of the input matrixes, and add these location information
when parallelizing the ParFor program body. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message