incubator-s4-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Gómez Ferro (JIRA) <j...@apache.org>
Subject [jira] [Commented] (S4-75) Code improvements in comm layer
Date Fri, 13 Jul 2012 15:51:35 GMT

    [ https://issues.apache.org/jira/browse/S4-75?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413834#comment-13413834
] 

Daniel Gómez Ferro commented on S4-75:
--------------------------------------

I've been having issues with the TCPEmitter. Lots of events were lost under high loads, here
are some example runs:
{code}
Successful: 47806  Lost: 241024
Successful: 220771 Lost: 68127
Successful: 53524  Lost: 235283
{code}

I tried a simpler implementation of the TCPEmitter (no queues, doesn't resend if an event
is lost...) and I didn't lose any event. 

I propose to substitute the current TCPEmitter, introduced in S4-7, with this simpler implementation.
It fixes these problems from the original JIRA:
{quote}
1. If the underlying topology changes, the channels and the associated connections are not
updated.
2. If a connection gets disconnected, it stays disconnected.
{quote}

But not this one:
{quote}
3. If for any reason, a connection can't be made, send() drops the message to be sent.
{quote}

I think it is worth it to lose some events on a connection failure instead of losing them
when all connections are working properly.

We can open a JIRA to reincorporate a queuing mechanism that fixes problem 3. in the future.
                
> Code improvements in comm layer
> -------------------------------
>
>                 Key: S4-75
>                 URL: https://issues.apache.org/jira/browse/S4-75
>             Project: Apache S4
>          Issue Type: Improvement
>    Affects Versions: 0.5
>            Reporter: Matthieu Morel
>             Fix For: 0.5
>
>
> After running into issues related to the comm layer, I took a deeper look and realized
parts of the code are really brittle or simply wrong, in particular in TCPEmitter (e.g. sync
on a Boolean).
> We need a more thorough review of that code so we can improve its robustness.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

Mime
View raw message