beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Luke Cwik (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (BEAM-2334) OutOfMemoryError in RandomAccessData.java:350
Date Sat, 20 May 2017 00:15:05 GMT

     [ https://issues.apache.org/jira/browse/BEAM-2334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Luke Cwik resolved BEAM-2334.
-----------------------------
       Resolution: Fixed
    Fix Version/s: 2.1.0

> OutOfMemoryError in RandomAccessData.java:350
> ---------------------------------------------
>
>                 Key: BEAM-2334
>                 URL: https://issues.apache.org/jira/browse/BEAM-2334
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-core
>    Affects Versions: 2.0.0
>            Reporter: Romain Yon
>            Assignee: Romain Yon
>             Fix For: 2.1.0
>
>
> Got the following trace:
> ```
> Caused by: java.lang.OutOfMemoryError: Requested array size exceeds VM limit
> 	at java.util.Arrays.copyOf(Arrays.java:3236)
> 	at org.apache.beam.runners.dataflow.util.RandomAccessData.ensureCapacity(RandomAccessData.java:350)
> 	at org.apache.beam.runners.dataflow.util.RandomAccessData.access$300(RandomAccessData.java:51)
> 	at org.apache.beam.runners.dataflow.util.RandomAccessData$1.write(RandomAccessData.java:258)
> 	at com.esotericsoftware.kryo.io.Output.flush(Output.java:185)
> ```
> However the machine was not out of heap space. Seems like the problem is in RandomAccessData.java:357:

> Arrays.copyOf(buffer, Integer.MAX_VALUE) 
> Proof:
> ```
> scala> Arrays.copyOf(v, Int.MaxValue)
> java.lang.OutOfMemoryError: Requested array size exceeds VM limit
>   at java.util.Arrays.copyOf(Arrays.java:3236)
>   ... 31 elided
> scala> Arrays.copyOf(v, Int.MaxValue-2)
> java.lang.OutOfMemoryError: Java heap space
>   at java.util.Arrays.copyOf(Arrays.java:3236)
>   ... 31 elided
> scala> Arrays.copyOf(v, Int.MaxValue-1)
> java.lang.OutOfMemoryError: Requested array size exceeds VM limit
>   at java.util.Arrays.copyOf(Arrays.java:3236)
>   ... 31 elided
> ```
> I'll submit a corresponding PR.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message