systemml-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias Boehm (JIRA)" <j...@apache.org>
Subject [jira] [Closed] (SYSTEMML-2026) Perftest OOM on genStratStatsData 80GB w/ codegen
Date Wed, 22 Nov 2017 23:24:00 GMT

     [ https://issues.apache.org/jira/browse/SYSTEMML-2026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Matthias Boehm closed SYSTEMML-2026.
------------------------------------
       Resolution: Fixed
         Assignee: Matthias Boehm
    Fix Version/s: SystemML 1.0

> Perftest OOM on genStratStatsData 80GB w/ codegen
> -------------------------------------------------
>
>                 Key: SYSTEMML-2026
>                 URL: https://issues.apache.org/jira/browse/SYSTEMML-2026
>             Project: SystemML
>          Issue Type: Bug
>            Reporter: Matthias Boehm
>            Assignee: Matthias Boehm
>             Fix For: SystemML 1.0
>
>
> {code}
> java.lang.OutOfMemoryError: Java heap space
> 	at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
> 	at java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
> 	at org.apache.spark.broadcast.TorrentBroadcast$$anonfun$3.apply(TorrentBroadcast.scala:271)
> 	at org.apache.spark.broadcast.TorrentBroadcast$$anonfun$3.apply(TorrentBroadcast.scala:271)
> 	at org.apache.spark.util.io.ChunkedByteBufferOutputStream.allocateNewChunkIfNeeded(ChunkedByteBufferOutputStream.scala:87)
> 	at org.apache.spark.util.io.ChunkedByteBufferOutputStream.write(ChunkedByteBufferOutputStream.scala:75)
> 	at net.jpountz.lz4.LZ4BlockOutputStream.flushBufferedData(LZ4BlockOutputStream.java:205)
> 	at net.jpountz.lz4.LZ4BlockOutputStream.finish(LZ4BlockOutputStream.java:235)
> 	at net.jpountz.lz4.LZ4BlockOutputStream.close(LZ4BlockOutputStream.java:175)
> 	at java.io.ObjectOutputStream$BlockDataOutputStream.close(ObjectOutputStream.java:1828)
> 	at java.io.ObjectOutputStream.close(ObjectOutputStream.java:742)
> 	at org.apache.spark.serializer.JavaSerializationStream.close(JavaSerializer.scala:57)
> 	at org.apache.spark.broadcast.TorrentBroadcast$$anonfun$blockifyObject$1.apply$mcV$sp(TorrentBroadcast.scala:278)
> 	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1346)
> 	at org.apache.spark.broadcast.TorrentBroadcast$.blockifyObject(TorrentBroadcast.scala:277)
> 	at org.apache.spark.broadcast.TorrentBroadcast.writeBlocks(TorrentBroadcast.scala:126)
> 	at org.apache.spark.broadcast.TorrentBroadcast.<init>(TorrentBroadcast.scala:88)
> 	at org.apache.spark.broadcast.TorrentBroadcastFactory.newBroadcast(TorrentBroadcastFactory.scala:34)
> 	at org.apache.spark.broadcast.BroadcastManager.newBroadcast(BroadcastManager.scala:56)
> 	at org.apache.spark.SparkContext.broadcast(SparkContext.scala:1488)
> 	at org.apache.spark.api.java.JavaSparkContext.broadcast(JavaSparkContext.scala:650)
> 	at org.apache.sysml.runtime.controlprogram.context.SparkExecutionContext.getBroadcastForVariable(SparkExecutionContext.java:559)
> 	at org.apache.sysml.runtime.instructions.spark.SpoofSPInstruction.processInstruction(SpoofSPInstruction.java:125)
> 	at org.apache.sysml.runtime.controlprogram.ProgramBlock.executeSingleInstruction(ProgramBlock.java:264)
> 	at org.apache.sysml.runtime.controlprogram.ProgramBlock.executeInstructions(ProgramBlock.java:218)
> 	at org.apache.sysml.runtime.controlprogram.ProgramBlock.execute(ProgramBlock.java:163)
> 	at org.apache.sysml.runtime.controlprogram.Program.execute(Program.java:118)
> 	at org.apache.sysml.api.ScriptExecutorUtils.executeRuntimeProgram(ScriptExecutorUtils.java:102)
> 	at org.apache.sysml.api.DMLScript.execute(DMLScript.java:747)
> 	at org.apache.sysml.api.DMLScript.executeScript(DMLScript.java:511)
> 	at org.apache.sysml.api.DMLScript.main(DMLScript.java:238)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message