flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fabian Hueske <fhue...@gmail.com>
Subject Re: [E] writeBufferLowWaterMark cannot be greater than writeBufferHighWaterMark error
Date Tue, 27 Dec 2016 15:00:33 GMT
Hi Kanagaraj,

I would assume that the issue is caused by this configuration parameter:

taskmanager.memory.segment-size: 131072

I think the maximum possible value given Netty's "writeBufferHighWaterMark"
parameter is 65536.
There might be a way to tune Netty's parameters but I don't know how to do
that. Maybe Ufuk (in CC) knows better.

Is there any particular reason why want to increase the network buffer size
instead of keeping the default?
Did you find that the default parameter gives insufficient performance?

Best, Fabian

2016-12-26 20:52 GMT+01:00 <kanagaraj.vengidasamy@verizon.com>:

>
> Hi,
>
> I am getting writeBufferLowWaterMark cannot be greater than
> writeBufferHighWaterMark  error frequently ...  and those task managers not
> processing messages after that error.
>  What could be wrong in my configuration?  What I need to do to avoid this
> error.?
>
> Have 8x32 VM's - 8 machines ( running 35 task managers - each one has 8
> slots)
> taskmanager.heap.mb: 4096
> taskmanager.numberOfTaskSlots: 8
> taskmanager.network.numberOfBuffers: 22000
> taskmanager.memory.segment-size: 131072
>
>
> 2016-12-26 14:26:06,548|  |WARN  io.netty.bootstrap.ServerBootstrap  -
> Failed to set a channel option: [id: 0xf1ef59e6, /138.83.31.4:60812 => /
> 138.83.31.9:41304]
> java.lang.IllegalArgumentException: writeBufferLowWaterMark cannot be
> greater than writeBufferHighWaterMark (65536): 131073
>         at io.netty.channel.DefaultChannelConfig.
> setWriteBufferLowWaterMark(DefaultChannelConfig.java:334)
>         at io.netty.channel.socket.DefaultSocketChannelConfig.
> setWriteBufferLowWaterMark(DefaultSocketChannelConfig.java:332)
>         at io.netty.channel.socket.DefaultSocketChannelConfig.
> setWriteBufferLowWaterMark(DefaultSocketChannelConfig.java:35)
>         at io.netty.channel.DefaultChannelConfig.setOption(
> DefaultChannelConfig.java:183)
>         at io.netty.channel.socket.DefaultSocketChannelConfig.setOption(
> DefaultSocketChannelConfig.java:121)
>         at io.netty.bootstrap.ServerBootstrap$ServerBootstrapAcceptor.
> channelRead(ServerBootstrap.java:238)
>         at io.netty.channel.AbstractChannelHandlerContext.
> invokeChannelRead(AbstractChannelHandlerContext.java:339)
>         at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(
> AbstractChannelHandlerContext.java:324)
>         at io.netty.channel.DefaultChannelPipeline.fireChannelRead(
> DefaultChannelPipeline.java:847)
>         at io.netty.channel.nio.AbstractNioMessageChannel$
> NioMessageUnsafe.read(AbstractNioMessageChannel.java:93)
>         at io.netty.channel.nio.NioEventLoop.processSelectedKey(
> NioEventLoop.java:511)
>         at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(
> NioEventLoop.java:468)
>         at io.netty.channel.nio.NioEventLoop.processSelectedKeys(
> NioEventLoop.java:382)
>         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
>         at io.netty.util.concurrent.SingleThreadEventExecutor$2.
> run(SingleThreadEventExecutor.java:111)
>         at java.lang.Thread.run(Thread.java:745)
>
>
> Thanks
> Kanagaraj Vengidasamy
> RTCI
>
> 7701 E Telecom PKWY
> Temple Terrace, FL 33637
>
> O 813.978.4372 | M 813.455.9757
>
>
>
>

Mime
View raw message