hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edward.y...@samsung.com>
Subject RE: Bug in Netty-based RPC
Date Fri, 08 May 2015 08:39:54 GMT
Simply reinstalled Ubuntu and now it works fine. :D

> Did you ever find the cause?

As I attached the link, it look like a issue on Netty side.

BTW, I feel like I need to explain my experiment here. Few weeks ago, I 
committed HAMA-954 so that you can turn on/off the message bundle behavior. If 
you set hama.bsp.messenger.bundle to true, the messenger system package the 
messages into single BSPMessageBundle object and send it at once. Meanwhile, 
if you set hama.bsp.messenger.bundle to false, the system send a message one 
by one asynchronously. I think the advantage of netty's non-blocking RPC can 
be maximized in this case (I couldn't test it yet).

--
Best Regards, Edward J. Yoon


-----Original Message-----
From: ByungSeok Min [mailto:byeongseok.min@gmail.com]
Sent: Friday, May 08, 2015 5:15 PM
To: dev@hama.apache.org
Subject: Re: Bug in Netty-based RPC

Hi~ Mr Yoon.

Is your machine ok?
Did you ever find the cause?


* My Work in progress is going to be finished well.
* I'll be back soon.


Best Regards!

2015-04-28 12:58 GMT+09:00 Edward J. Yoon <edwardyoon@apache.org>:

> I just found this
>
> https://github.com/netty/netty/commit/dcf78961724abf97fea9e768cca02ba65146b324
>
> On Tue, Apr 28, 2015 at 11:39 AM, Chia-Hung Lin <clin4j@googlemail.com>
> wrote:
> > Have you checked limit.conf?
> >
> > From the message it looks like the files opened at underlying system
> > exceed its default limit.
> >
> > On 28 April 2015 at 08:08, Edward J. Yoon <edwardyoon@apache.org> wrote:
> >> I tried to run BSP job using netty-based RPC instead of message
> >> bundle, but I received "too many open files".
> >>
> >> --
> >>
> >> attempt_201504280858_0001_000017_0: 15/04/28 08:28:17 INFO
> >> ipc.AsyncClient: AsyncClient startup
> >> attempt_201504280858_0001_000017_0: 15/04/28 08:28:21 ERROR
> >> bsp.BSPTask: Error running bsp setup and bsp function.
> >> attempt_201504280858_0001_000017_0: java.lang.IllegalStateException:
> >> failed to create a child event loop
> >> attempt_201504280858_0001_000017_0: at
> >>
> io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:68)
> >> attempt_201504280858_0001_000017_0: at
> >>
> io.netty.channel.MultithreadEventLoopGroup.<init>(MultithreadEventLoopGroup.java:49)
> >> attempt_201504280858_0001_000017_0: at
> >> io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:61)
> >> attempt_201504280858_0001_000017_0: at
> >> io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:52)
> >> attempt_201504280858_0001_000017_0: at
> >> io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:44)
> >> attempt_201504280858_0001_000017_0: at
> >> io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:36)
> >> attempt_201504280858_0001_000017_0: at
> >> org.apache.hama.ipc.AsyncClient$Connection.<init>(AsyncClient.java:189)
> >> attempt_201504280858_0001_000017_0: at
> >> org.apache.hama.ipc.AsyncClient.getConnection(AsyncClient.java:989)
> >> attempt_201504280858_0001_000017_0: at
> >> org.apache.hama.ipc.AsyncClient.call(AsyncClient.java:838)
> >> attempt_201504280858_0001_000017_0: at
> >> org.apache.hama.ipc.AsyncRPC$Invoker.invoke(AsyncRPC.java:261)
> >> attempt_201504280858_0001_000017_0: at
> >> com.sun.proxy.$Proxy14.getProtocolVersion(Unknown Source)
> >> attempt_201504280858_0001_000017_0: at
> >> org.apache.hama.ipc.AsyncRPC.checkVersion(AsyncRPC.java:524)
> >> attempt_201504280858_0001_000017_0: at
> >> org.apache.hama.ipc.AsyncRPC.getProxy(AsyncRPC.java:509)
> >> attempt_201504280858_0001_000017_0: at
> >> org.apache.hama.ipc.AsyncRPC.getProxy(AsyncRPC.java:477)
> >> attempt_201504280858_0001_000017_0: at
> >> org.apache.hama.ipc.AsyncRPC.getProxy(AsyncRPC.java:435)
> >> attempt_201504280858_0001_000017_0: at
> >> org.apache.hama.ipc.AsyncRPC.getProxy(AsyncRPC.java:545)
> >> attempt_201504280858_0001_000017_0: at
> >>
> org.apache.hama.bsp.message.HamaAsyncMessageManagerImpl.getBSPPeerConnection(HamaAsyncMessageManagerImpl.java:155)
> >> attempt_201504280858_0001_000017_0: at
> >>
> org.apache.hama.bsp.message.HamaAsyncMessageManagerImpl.transfer(HamaAsyncMessageManagerImpl.java:203)
> >> attempt_201504280858_0001_000017_0: at
> >> org.apache.hama.bsp.BSPPeerImpl.sendDirectly(BSPPeerImpl.java:382)
> >> attempt_201504280858_0001_000017_0: at
> >> org.apache.hama.bsp.BSPPeerImpl.send(BSPPeerImpl.java:364)
> >> attempt_201504280858_0001_000017_0: at
> >>
> org.apache.hama.graph.GraphJobRunner.loadVertices(GraphJobRunner.java:467)
> >> attempt_201504280858_0001_000017_0: at
> >> org.apache.hama.graph.GraphJobRunner.setup(GraphJobRunner.java:128)
> >> attempt_201504280858_0001_000017_0: at
> >> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:170)
> >> attempt_201504280858_0001_000017_0: at
> >> org.apache.hama.bsp.BSPTask.run(BSPTask.java:144)
> >> attempt_201504280858_0001_000017_0: at
> >> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1255)
> >> attempt_201504280858_0001_000017_0: Caused by:
> >> io.netty.channel.ChannelException: failed to open a new selector
> >> attempt_201504280858_0001_000017_0: at
> >> io.netty.channel.nio.NioEventLoop.openSelector(NioEventLoop.java:128)
> >> attempt_201504280858_0001_000017_0: at
> >> io.netty.channel.nio.NioEventLoop.<init>(NioEventLoop.java:120)
> >> attempt_201504280858_0001_000017_0: at
> >>
> io.netty.channel.nio.NioEventLoopGroup.newChild(NioEventLoopGroup.java:87)
> >> attempt_201504280858_0001_000017_0: at
> >>
> io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:64)
> >> attempt_201504280858_0001_000017_0: ... 24 more
> >> attempt_201504280858_0001_000017_0: Caused by: java.io.IOException:
> >> Too many open files
> >> attempt_201504280858_0001_000017_0: at
> sun.nio.ch.IOUtil.makePipe(Native Method)
> >> attempt_201504280858_0001_000017_0: at
> >> sun.nio.ch.EPollSelectorImpl.<init>(EPollSelectorImpl.java:65)
> >> attempt_201504280858_0001_000017_0: at
> >>
> sun.nio.ch.EPollSelectorProvider.openSelector(EPollSelectorProvider.java:36)
> >> attempt_201504280858_0001_000017_0: at
> >> io.netty.channel.nio.NioEventLoop.openSelector(NioEventLoop.java:126)
> >> attempt_201504280858_0001_000017_0: ... 27 more
> >> attempt_201504280858_0001_000017_0: 15/04/28 08:28:21 INFO
> >> ipc.AsyncServer: AsyncServer gracefully shutdown
> >>
> >>
> >> --
> >> Best Regards, Edward J. Yoon
>
>
>
> --
> Best Regards, Edward J. Yoon
>



Mime
View raw message