flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andra Lungu <lungu.an...@gmail.com>
Subject Memory segment error
Date Thu, 26 Mar 2015 15:46:21 GMT
Hello everyone,

I guess I need to revive this old discussion:
http://apache-flink-incubator-mailing-list-archive.1008284.n3.nabble.com/Memory-segment-error-when-migrating-functional-code-from-Flink-0-9-to-0-8-td3687.html

At that point, the fix was to kindly ask Alex to make his project work with
0.9.

Now, I am not that lucky!

This is the code:
https://github.com/andralungu/gelly-partitioning/tree/alphaSplit

The main program(NodeSplitting) is working nicely, I get the correct
result. But if you run the test,  you will see that collection works and
cluster fails miserably with this exception:

Caused by: java.lang.Exception: The data preparation for task 'Join(Join at
weighEdges(NodeSplitting.java:112)) (04e172e761148a65783a4363406e08c0)' ,
caused an error: Too few memory segments provided. Hash Join needs at least
33 memory segments.
    at
org.apache.flink.runtime.operators.RegularPactTask.run(RegularPactTask.java:471)
    at
org.apache.flink.runtime.operators.RegularPactTask.invoke(RegularPactTask.java:362)
    at
org.apache.flink.runtime.execution.RuntimeEnvironment.run(RuntimeEnvironment.java:209)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Too few memory segments
provided. Hash Join needs at least 33 memory segments.

I am running locally, from IntelliJ, on a tiny graph.
$ cat /proc/meminfo
MemTotal:       11405696 kB
MemFree:         5586012 kB
Buffers:          178100 kB

I am sure I did not run out of memory...

Any thoughts on this?

Thanks!
Andra

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message