From issues-return-9118-archive-asf-public=cust-asf.ponee.io@systemml.apache.org Sat Apr 13 08:22:02 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id CC15D180668 for ; Sat, 13 Apr 2019 10:22:01 +0200 (CEST) Received: (qmail 94442 invoked by uid 500); 13 Apr 2019 08:22:01 -0000 Mailing-List: contact issues-help@systemml.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@systemml.apache.org Delivered-To: mailing list issues@systemml.apache.org Received: (qmail 94417 invoked by uid 99); 13 Apr 2019 08:22:01 -0000 Received: from mailrelay1-us-west.apache.org (HELO mailrelay1-us-west.apache.org) (209.188.14.139) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 13 Apr 2019 08:22:00 +0000 Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 7023AE28A0 for ; Sat, 13 Apr 2019 08:22:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 2D16724599 for ; Sat, 13 Apr 2019 08:22:00 +0000 (UTC) Date: Sat, 13 Apr 2019 08:22:00 +0000 (UTC) From: "Janardhan (JIRA)" To: issues@systemml.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Assigned] (SYSTEMML-1158) MLContext values not found when data is read in DML script MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/SYSTEMML-1158?page=3Dcom.atlas= sian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhan reassigned SYSTEMML-1158: ----------------------------------- Assignee: Janardhan > MLContext values not found when data is read in DML script > ---------------------------------------------------------- > > Key: SYSTEMML-1158 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1158 > Project: SystemML > Issue Type: Bug > Components: APIs > Affects Versions: SystemML 0.11 > Environment: IBM DSX > Reporter: Felix Sch=C3=BCler > Assignee: Janardhan > Priority: Major > > When executing a DML script through MLContext that reads data from a file= , resulting variables can not be found in the MLContext because they are re= moved from the context. This does not happen when the data is initialized r= andomly inside the script.=20 > This example works fine with MLContext.getTuple > {code} > data =3D rand(100, 10) > c =3D ncol(data) > r =3D nrow(data) > stats =3D rand(rows=3D1, cols=3Dc) > for (i in 0:(r - 1)) { > stats =3D (stats + data[i + 1,]) > } > stats =3D (stats / r) > {code} > When I replace the random matrix initialization with {code} read("~/abalo= ne_transformed", format=3D"csv"){code} I get the following exception:=20 > {code}=20 > Name: org.apache.sysml.api.mlcontext.MLContextException > Message: Variable 'stats' not found > StackTrace: at org.apache.sysml.api.mlcontext.MLResults.getData(MLResul= ts.java:103) > at org.apache.sysml.api.mlcontext.MLResults.outputValue(MLResults.java:= 1996) > at org.apache.sysml.api.mlcontext.MLResults.getTuple(MLResults.java:663= ) > {code} > The instructions for the working case: > {code} > PROGRAM ( size CP/SP =3D 28/0 ) > --MAIN PROGRAM > ----GENERIC (lines 1-4) [recompile=3Dfalse] > ------CP createvar _mVar167234 scratch_space//_p664_10.143.133.52//_t0/te= mp83636 true MATRIX binaryblock 3 3 1000 1000 9 copy > ------CP rand 3 3 1000 1000 0.0 1.0 1.0 -1 uniform 1.0 48 _mVar167234.MAT= RIX.DOUBLE > ------CP createvar _mVar167235 scratch_space//_p664_10.143.133.52//_t0/te= mp83637 true MATRIX binaryblock 1 3 1000 1000 3 copy > ------CP rand 1 3 1000 1000 0.0 1.0 1.0 -1 uniform 1.0 48 _mVar167235.MAT= RIX.DOUBLE > ------CP assignvar 3.SCALAR.INT.true c.SCALAR.INT > ------CP assignvar 3.SCALAR.INT.true r.SCALAR.INT > ------CP cpvar _mVar167234 data > ------CP cpvar _mVar167235 stats > ------CP rmvar _mVar167234 > ------CP rmvar _mVar167235 > ----GENERIC (lines 5-7) [recompile=3Dfalse] > ----FOR (lines 5-7) > ------GENERIC (lines 6-6) [recompile=3Dfalse] > --------CP + i.SCALAR.INT.false 1.SCALAR.INT.true _Var167236.SCALAR.INT > --------CP createvar _mVar167237 scratch_space//_p664_10.143.133.52//_t0/= temp83638 true MATRIX binaryblock 1 3 1000 1000 -1 copy > --------CP rangeReIndex data.MATRIX.DOUBLE _Var167236.SCALAR.INT.false _V= ar167236.SCALAR.INT.false 1.SCALAR.INT.true 3.SCALAR.INT.true _mVar167237.M= ATRIX.DOUBLE > --------CP rmvar _Var167236 > --------CP createvar _mVar167238 scratch_space//_p664_10.143.133.52//_t0/= temp83639 true MATRIX binaryblock 1 3 1000 1000 -1 copy > --------CP + stats.MATRIX.DOUBLE _mVar167237.MATRIX.DOUBLE _mVar167238.MA= TRIX.DOUBLE > --------CP rmvar _mVar167237 > --------CP rmvar stats > --------CP cpvar _mVar167238 stats > --------CP rmvar _mVar167238 > ----GENERIC (lines 8-11) [recompile=3Dfalse] > ------CP createvar _mVar167239 scratch_space//_p664_10.143.133.52//_t0/te= mp83640 true MATRIX binaryblock 1 3 1000 1000 -1 copy > ------CP / stats.MATRIX.DOUBLE 3.SCALAR.INT.true _mVar167239.MATRIX.DOUBL= E > ------CP rmvar stats > ------CP cpvar _mVar167239 stats > ------CP rmvar _mVar167239 > ------CP rmvar r > ------CP rmvar c > ------CP rmvar stats > {code} > Failing case (note that this is in a notebook reading a file from object = storage) > {code} > PROGRAM ( size CP/SP =3D 38/7 ) > --MAIN PROGRAM > ----GENERIC (lines 1-4) [recompile=3Dtrue] > ------CP createvar pREADdata ~/abalone_transformed false MATRIX csv -1 -1= -1 -1 -1 copy false , true 0.0 > ------CP createvar _mVar150511 scratch_space//_p664_10.143.133.52//_t0/te= mp75273 true MATRIX binaryblock -1 -1 1000 1000 -1 copy > ------SPARK csvrblk pREADdata.MATRIX.DOUBLE _mVar150511.MATRIX.DOUBLE 100= 0 1000 false , true 0.0 > ------CP createvar _mVar150512 scratch_space//_p664_10.143.133.52//_t0/te= mp75274 true MATRIX binaryblock -1 -1 1000 1000 -1 copy > ------SPARK chkpoint _mVar150511.MATRIX.DOUBLE _mVar150512.MATRIX.DOUBLE = MEMORY_AND_DISK > ------CP rmvar _mVar150511 > ------CP cpvar _mVar150512 data > ------CP rmvar _mVar150512 > ----GENERIC (lines 1-4) [recompile=3Dtrue] > ------CP ncol data.MATRIX.DOUBLE.false _Var150513.SCALAR.INT > ------CP nrow data.MATRIX.DOUBLE.false _Var150514.SCALAR.INT > ------CP createvar _mVar150515 scratch_space//_p664_10.143.133.52//_t0/te= mp75275 true MATRIX binaryblock 1 -1 1000 1000 -1 copy > ------SPARK rand 1 =C2=B6_Var150513=C2=B6 1000 1000 0.0 1.0 1.0 -1 scratc= h_space/_p664_10.143.133.52//_t0/ uniform 1.0 _mVar150515.MATRIX.DOUBLE > ------CP assignvar _Var150513.SCALAR.INT.false c.SCALAR.INT > ------CP assignvar _Var150514.SCALAR.INT.false r.SCALAR.INT > ------CP rmvar _Var150513 > ------CP rmvar _Var150514 > ------CP cpvar _mVar150515 stats > ------CP rmvar _mVar150515 > ----GENERIC (lines 5-7) [recompile=3Dtrue] > ------CP createvar _mVar150516 scratch_space//_p664_10.143.133.52//_t0/te= mp75276 true MATRIX binaryblock -1 -1 1000 1000 -1 copy > ------SPARK chkpoint data.MATRIX.DOUBLE _mVar150516.MATRIX.DOUBLE MEMORY_= AND_DISK > ------CP rmvar data > ------CP cpvar _mVar150516 data > ------CP rmvar _mVar150516 > ----FOR (lines 5-7) > ------CP - r.SCALAR.INT.false 1.SCALAR.INT.true _Var150517.SCALAR.INT > ------CP rmvar _Var150517 > ------GENERIC (lines 6-6) [recompile=3Dtrue] > --------CP + i.SCALAR.INT.false 1.SCALAR.INT.true _Var150518.SCALAR.INT > --------CP ncol data.MATRIX.DOUBLE.false _Var150519.SCALAR.INT > --------CP createvar _mVar150520 scratch_space//_p664_10.143.133.52//_t0/= temp75277 true MATRIX binaryblock 1 -1 1000 1000 -1 copy > --------SPARK rangeReIndex data.MATRIX.DOUBLE _Var150518.SCALAR.INT.false= _Var150518.SCALAR.INT.false 1.SCALAR.INT.true _Var150519.SCALAR.INT.false = _mVar150520.MATRIX.DOUBLE MULTI_BLOCK > --------CP rmvar _Var150518 > --------CP rmvar _Var150519 > --------CP createvar _mVar150521 scratch_space//_p664_10.143.133.52//_t0/= temp75278 true MATRIX binaryblock 1 -1 1000 1000 -1 copy > --------SPARK + stats.MATRIX.DOUBLE _mVar150520.MATRIX.DOUBLE _mVar150521= .MATRIX.DOUBLE > --------CP rmvar _mVar150520 > --------CP rmvar stats > --------CP cpvar _mVar150521 stats > --------CP rmvar _mVar150521 > ----GENERIC (lines 8-11) [recompile=3Dtrue] > ------CP createvar _mVar150522 scratch_space//_p664_10.143.133.52//_t0/te= mp75279 true MATRIX binaryblock 1 -1 1000 1000 -1 copy > ------SPARK / stats.MATRIX.DOUBLE r.SCALAR.INT.false _mVar150522.MATRIX.D= OUBLE > ------CP rmvar stats > ------CP cpvar _mVar150522 stats > ------CP rmvar _mVar150522 > ------CP rmvar r > ------CP rmvar c > ------CP rmvar stats > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)