systemml-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias Boehm (JIRA)" <j...@apache.org>
Subject [jira] [Closed] (SYSTEMML-1884) Unnecessary spark/mr instructions on removeEmpty select
Date Fri, 08 Sep 2017 19:32:01 GMT

     [ https://issues.apache.org/jira/browse/SYSTEMML-1884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Matthias Boehm closed SYSTEMML-1884.
------------------------------------

> Unnecessary spark/mr instructions on removeEmpty select
> -------------------------------------------------------
>
>                 Key: SYSTEMML-1884
>                 URL: https://issues.apache.org/jira/browse/SYSTEMML-1884
>             Project: SystemML
>          Issue Type: Bug
>    Affects Versions: SystemML 0.14
>            Reporter: Matthias Boehm
>            Assignee: Matthias Boehm
>             Fix For: SystemML 0.15
>
>
> The builtin function {{removeEmpty}} takes an optional select parameter. If this parameter
is provided, we directly use it instead of explicitly computing it. However, the compilation
of removeEmpty handled this incorrectly, setting the forced execution type to the initially
compiled execution type of removeEmpty. This is problematic in scenarios with dynamic recompilation.
For example, consider the following scenario:
> {code}
> C = replace(target=A, pattern=2, replacement=0);
> D = removeEmpty(B, margin="rows", select=C);
> {code}
> If the {{removeEmpty}} is initially compiled to SPARK, this also mistakenly set the forced
execution type of {{replace}} to SPARK. During recompilation, the {{removeEmpty}} might be
compiled to CP but because {{replace}} has a forced execution type, it will remain in SPARK.

> Workaround: The workaround for this issue is to put a {{while(FALSE){}}} just before
the {{removeEmpty}} call (more precisely, C in above example needs to be computed in a separate
HOP DAG).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message