flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sourigna Phetsarath <gna.phetsar...@teamaol.com>
Subject Setting taskmanager.network.numberOfBuffers and getting errors...
Date Thu, 03 Mar 2016 21:27:25 GMT
All:

I'm running a Flink 0.10.2 App by submitting to YARN as an application.
I'm using an AWS EMR cluster of 1 Master and 10 d2.8xlarge.  When I submit
the job using:


bin/flink run \
    -m yarn-cluster \
    -yjm 20480 \
    -yn 10 \
    -ytm 80960 \
    -ys 36 \
    -yD taskmanager.network.numberOfBuffers=*51840* \
...

I'm seeing this error:

Caused by: java.io.IOException: Insufficient number of network buffers:
required *360*, but only *315* available. The total number of network
buffers is currently set to *51840*. You can increase this number by
setting the configuration key '*taskmanager.network.numberOfBuffers*'.

at
org.apache.flink.runtime.io.network.buffer.NetworkBufferPool.createBufferPool(NetworkBufferPool.java:196)

at
org.apache.flink.runtime.io.network.NetworkEnvironment.registerTask(NetworkEnvironment.java:325)

at org.apache.flink.runtime.taskmanager.Task.run(Task.java:488)

at java.lang.Thread.run(Thread.java:745)

The error message does not seem to be conveying the correct information.

Can someone explain to me, what are reasonable numbers to use for
*taskmanager.network.numberOfBuffers* and *t*
*askmanager.network.bufferSizeInBytes*

I've read this:
https://ci.apache.org/projects/flink/flink-docs-master/setup/config.html#configuring-the-network-buffers
and this:
http://stackoverflow.com/questions/33589710/flink-cluster-params-how-to-set

But I am still unclear of the calculus is it supposed to be?

 #cores ^ 2 * #machines * 4

So, in my case  36 ^ 2 * 10 * 4 = 51840

Thanks in advance for you help that you can provide.

-- 


*Gna Phetsarath*System Architect // AOL Platforms // Data Services //
Applied Research Chapter
770 Broadway, 5th Floor, New York, NY 10003
o: 212.402.4871 // m: 917.373.7363
vvmr: 8890237 aim: sphetsarath20 t: @sourigna

* <http://www.aolplatforms.com>*

Mime
View raw message