systemml-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Glenn Weidner (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SYSTEMML-1588) Perftest: Datagen descriptive statistics fails on parfor result merge
Date Sat, 09 Sep 2017 01:11:00 GMT

     [ https://issues.apache.org/jira/browse/SYSTEMML-1588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Glenn Weidner updated SYSTEMML-1588:
------------------------------------
    Fix Version/s:     (was: SystemML 1.0)
                   SystemML 0.15

> Perftest: Datagen descriptive statistics fails on parfor result merge
> ---------------------------------------------------------------------
>
>                 Key: SYSTEMML-1588
>                 URL: https://issues.apache.org/jira/browse/SYSTEMML-1588
>             Project: SystemML
>          Issue Type: Bug
>    Affects Versions: SystemML 0.14
>            Reporter: Matthias Boehm
>            Assignee: Matthias Boehm
>             Fix For: SystemML 0.15
>
>
> The perftest data generation script for descriptive statistics fails for the 80GB scenario
with the following stacktrace
> {code}
> Caused by: org.apache.sysml.runtime.DMLRuntimeException: org.apache.sysml.runtime.DMLRuntimeException:
org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: hdfs://<node>:<port>/user/biuser/scratch_space/_p123456789_1.2.34.56/_t16/temp863_830
> 	at org.apache.sysml.runtime.controlprogram.parfor.ResultMergeRemoteSpark.executeParallelMerge(ResultMergeRemoteSpark.java:116)
> 	at org.apache.sysml.runtime.controlprogram.parfor.ResultMergeRemoteSpark.executeSerialMerge(ResultMergeRemoteSpark.java:71)
> 	at org.apache.sysml.runtime.controlprogram.ParForProgramBlock.consolidateAndCheckResults(ParForProgramBlock.java:1756)
> 	at org.apache.sysml.runtime.controlprogram.ParForProgramBlock.executeLocalParFor(ParForProgramBlock.java:818)
> 	at org.apache.sysml.runtime.controlprogram.ParForProgramBlock.execute(ParForProgramBlock.java:639)
> 	... 14 more
> {code}
> This issue is caused by incorrect export handling of inputs which only works for existing
inputs on hdfs or dirty in-memory objects but not for pending rdd operations.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message