systemml-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias Boehm (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SYSTEMML-2152) Spark mapmm fails for broadcast with larger than integer dims
Date Sat, 17 Feb 2018 01:12:03 GMT
Matthias Boehm created SYSTEMML-2152:
----------------------------------------

             Summary: Spark mapmm fails for broadcast with larger than integer dims
                 Key: SYSTEMML-2152
                 URL: https://issues.apache.org/jira/browse/SYSTEMML-2152
             Project: SystemML
          Issue Type: Bug
            Reporter: Matthias Boehm


{code}

        at org.apache.sysml.runtime.controlprogram.context.SparkExecutionContext.getBroadcastForVariable(SparkExecutionContext.java:545)
        at org.apache.sysml.runtime.instructions.spark.MapmmSPInstruction.processInstruction(MapmmSPInstruction.java:128)
        at org.apache.sysml.runtime.controlprogram.ProgramBlock.executeSingleInstruction(ProgramBlock.java:264)
        ... 17 more
Caused by: java.io.IOException: org.apache.sysml.runtime.DMLRuntimeException: Matrix dimensions
too large for CP runtime: 514 x 5129281161
        at org.apache.sysml.runtime.util.DataConverter.readMatrixFromHDFS(DataConverter.java:211)
        at org.apache.sysml.runtime.util.DataConverter.readMatrixFromHDFS(DataConverter.java:171)
        at org.apache.sysml.runtime.controlprogram.caching.MatrixObject.readBlobFromHDFS(MatrixObject.java:434)
        at org.apache.sysml.runtime.controlprogram.caching.MatrixObject.readBlobFromHDFS(MatrixObject.java:59)
        at org.apache.sysml.runtime.controlprogram.caching.CacheableData.readBlobFromHDFS(CacheableData.java:981)
        at org.apache.sysml.runtime.controlprogram.caching.CacheableData.acquireRead(CacheableData.java:417)
        ... 20 more
Caused by: org.apache.sysml.runtime.DMLRuntimeException: Matrix dimensions too large for CP
runtime: 514 x 5129281161
        at org.apache.sysml.runtime.io.MatrixReader.createOutputMatrixBlock(MatrixReader.java:80)
        at org.apache.sysml.runtime.io.ReaderBinaryBlockParallel.readMatrixFromHDFS(ReaderBinaryBlockParallel.java:59)
        at org.apache.sysml.runtime.util.DataConverter.readMatrixFromHDFS(DataConverter.java:207)
        ... 25 more
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message