flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guido <gmazza...@gmail.com>
Subject Insufficient number of network buffers running on cluster
Date Fri, 27 Nov 2015 15:41:47 GMT

I would like to ask you something regarding an error I’m facing running java code over the
cluster at DIMA.

Caused by: java.io.IOException: Insufficient number of network buffers: required 300, but
only 161 available. The total number of network buffers is currently set to 70480. You can
increase this number by setting the configuration key 'taskmanager.network.numberOfBuffers'.
	at org.apache.flink.runtime.io.network.buffer.NetworkBufferPool.createBufferPool(NetworkBufferPool.java:176)
	at org.apache.flink.runtime.io.network.NetworkEnvironment.registerTask(NetworkEnvironment.java:313)
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:464)
	at java.lang.Thread.run(Thread.java:853)

The error seems pretty clear so as suggested, I increased  the taskmanager.network.numberOfBuffers
from the flink-conf.yaml, (first was setted to 35240), but actually this didn’t solve the
problem, and the number of the available buffers is almost always the same.
Could you explain about some further configurations I should check above the cluster?
Besides solve this issue I would be interested in understanding why I get this and where these
values come from.

Thanks in advance for your time,

View raw message