flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Metzger <rmetz...@apache.org>
Subject Re: Memory segment error
Date Thu, 26 Mar 2015 15:48:46 GMT
Hi,

during startup, Flink will log something like:
16:48:09,669 INFO  org.apache.flink.runtime.taskmanager.TaskManager
     - Using 1193 MB for Flink managed memory.

Can you tell us how much memory Flink is managing in your case?



On Thu, Mar 26, 2015 at 4:46 PM, Andra Lungu <lungu.andra@gmail.com> wrote:

> Hello everyone,
>
> I guess I need to revive this old discussion:
>
> http://apache-flink-incubator-mailing-list-archive.1008284.n3.nabble.com/Memory-segment-error-when-migrating-functional-code-from-Flink-0-9-to-0-8-td3687.html
>
> At that point, the fix was to kindly ask Alex to make his project work with
> 0.9.
>
> Now, I am not that lucky!
>
> This is the code:
> https://github.com/andralungu/gelly-partitioning/tree/alphaSplit
>
> The main program(NodeSplitting) is working nicely, I get the correct
> result. But if you run the test,  you will see that collection works and
> cluster fails miserably with this exception:
>
> Caused by: java.lang.Exception: The data preparation for task 'Join(Join at
> weighEdges(NodeSplitting.java:112)) (04e172e761148a65783a4363406e08c0)' ,
> caused an error: Too few memory segments provided. Hash Join needs at least
> 33 memory segments.
>     at
>
> org.apache.flink.runtime.operators.RegularPactTask.run(RegularPactTask.java:471)
>     at
>
> org.apache.flink.runtime.operators.RegularPactTask.invoke(RegularPactTask.java:362)
>     at
>
> org.apache.flink.runtime.execution.RuntimeEnvironment.run(RuntimeEnvironment.java:209)
>     at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.IllegalArgumentException: Too few memory segments
> provided. Hash Join needs at least 33 memory segments.
>
> I am running locally, from IntelliJ, on a tiny graph.
> $ cat /proc/meminfo
> MemTotal:       11405696 kB
> MemFree:         5586012 kB
> Buffers:          178100 kB
>
> I am sure I did not run out of memory...
>
> Any thoughts on this?
>
> Thanks!
> Andra
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message