avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jan Van Besien (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AVRO-1391) NettyTransceiver await called in IO thread
Date Tue, 05 Nov 2013 07:43:20 GMT

    [ https://issues.apache.org/jira/browse/AVRO-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13813724#comment-13813724

Jan Van Besien commented on AVRO-1391:

FYI. I worked around the problem by starting up new threads for the (avro-ipc) communication
with my backend system from within the custom flume sink. Although the whole flow is still
synchronous (I start a thread to do the synchronous call and wait for the thread to finish),
Netty no longer thinks that the call originates from an IO thread.

The deadlock detection mechanism in Netty is wrong here I think, because the IO thread is
actually only an IO thread on a different Netty pipeline.

If the NettyTransceiver implementation in Avro would be truly asynchronous (i.e. do not wait
for the channel to be created but add a listener which gets notified when it is ready), the
problem would not occur. However, I can imagine that this would be a non trivial change which
would ripple up in client code.

> NettyTransceiver await called in IO thread
> ------------------------------------------
>                 Key: AVRO-1391
>                 URL: https://issues.apache.org/jira/browse/AVRO-1391
>             Project: Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.7.5
>            Reporter: Jan Van Besien
> The NettyTransceiver does a call to channelFuture.await() in the getChannel() method
(line 271). This fails with the following exception if avro ipc is used from within an IO
thread from another avro-ipc call.
> Caused by: java.lang.IllegalStateException: await*() in I/O thread causes a dead lock
or sudden performance drop. Use addListener() instead or call await*() from a different thread.
>         at org.jboss.netty.channel.DefaultChannelFuture.checkDeadLock(DefaultChannelFuture.java:314)
>         at org.jboss.netty.channel.DefaultChannelFuture.await0(DefaultChannelFuture.java:278)
> ...
> The use case where I was able to reproduce this is when apache flume's avro source is
coupled with a custom flume sink that also uses avro-ipc to talk to something else. In that
case, the thread that ultimately does the above getChannel() call is an "avro io thread".

This message was sent by Atlassian JIRA

View raw message