incubator-s4-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthieu Morel (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (S4-7) Netty to tolerate network glitches and connection loss
Date Wed, 07 Dec 2011 16:38:40 GMT

    [ https://issues.apache.org/jira/browse/S4-7?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164497#comment-13164497
] 

Matthieu Morel commented on S4-7:
---------------------------------

Karthik's answer (30th) nov, on the dev list:

{quote}
Yes, we might loose messages on the wire and I tried to address it in my
previous version (that did not work). However, the ChannelFuture.success
was not behaving as "expected". I need to go back to the board and figure
the right way to do it. I can create a JIRA for that.

Regarding the parameters, I would let experienced people like Kishore,
Bruce and you pick the default values - I ll make them tunable along with
the above change.

{quote}


                
> Netty to tolerate network glitches and connection loss
> ------------------------------------------------------
>
>                 Key: S4-7
>                 URL: https://issues.apache.org/jira/browse/S4-7
>             Project: Apache S4
>          Issue Type: Bug
>            Reporter: Leo Neumeyer
>            Assignee: Karthik Kambatla
>             Fix For: 0.5
>
>
> NettyEmitter connects to different partitions and creates channels over which it communicates
to other listeners.
> It suffers from the following issues -- 
> 1. If the underlying topology changes, the channels and the associated connections are
not updated.
> 2. If a connection gets disconnected, it stays disconnected.
> 3. If for any reason, a connection can't be made, send() drops the message to be sent.
> The solution is to - 
> 1. Maintain a bounded messageQueue for each destination partition - if a connection does
not exist, the message should be queued.
> 2. Maintain a map of the channel used for each destination partition - update this map
on changes to topology, or on send() in case of disconnections.
> 3. Every time a (re-)connection is made, send the queued messages first.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message