beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Romain Yon (JIRA)" <j...@apache.org>
Subject [jira] [Created] (BEAM-2334) OutOfMemoryError in RandomAccessData.java:350
Date Fri, 19 May 2017 20:15:05 GMT
Romain Yon created BEAM-2334:
--------------------------------

             Summary: OutOfMemoryError in RandomAccessData.java:350
                 Key: BEAM-2334
                 URL: https://issues.apache.org/jira/browse/BEAM-2334
             Project: Beam
          Issue Type: Bug
          Components: runner-core
    Affects Versions: 2.0.0
            Reporter: Romain Yon
            Assignee: Kenneth Knowles


Got the following trace:

```
Caused by: java.lang.OutOfMemoryError: Requested array size exceeds VM limit
	at java.util.Arrays.copyOf(Arrays.java:3236)
	at org.apache.beam.runners.dataflow.util.RandomAccessData.ensureCapacity(RandomAccessData.java:350)
	at org.apache.beam.runners.dataflow.util.RandomAccessData.access$300(RandomAccessData.java:51)
	at org.apache.beam.runners.dataflow.util.RandomAccessData$1.write(RandomAccessData.java:258)
	at com.esotericsoftware.kryo.io.Output.flush(Output.java:185)
```

However the machine was not out of heap space. Seems like the problem is in RandomAccessData.java:357:

Arrays.copyOf(buffer, Integer.MAX_VALUE) 

Proof:
```
scala> Arrays.copyOf(v, Int.MaxValue)
java.lang.OutOfMemoryError: Requested array size exceeds VM limit
  at java.util.Arrays.copyOf(Arrays.java:3236)
  ... 31 elided

scala> Arrays.copyOf(v, Int.MaxValue-2)
java.lang.OutOfMemoryError: Java heap space
  at java.util.Arrays.copyOf(Arrays.java:3236)
  ... 31 elided

scala> Arrays.copyOf(v, Int.MaxValue-1)
java.lang.OutOfMemoryError: Requested array size exceeds VM limit
  at java.util.Arrays.copyOf(Arrays.java:3236)
  ... 31 elided
```

I'll submit a corresponding PR.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message