cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sylvain Lebresne (Commented) (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-3569) Failure detector downs should not break streams
Date Thu, 08 Dec 2011 10:18:40 GMT


Sylvain Lebresne commented on CASSANDRA-3569:

bq. But given that we are using TCP, it's not adding anything, it seems to me, except for
the fact that the FD is more tweakable (from the app) than the keep-alive timeouts.

That's not the definition of being orthogonal in my book, but ok. I'll also note that the
FD mostly pretend to adapt to the network condition (whether it does that well or not is another
question). I don't think anybody ever pretended it was handling the connectivity to other
nodes better that TCP.  Besides, the FD is also here to detect node failures even when we're
not actively communicating with them. But yes, this is not the case for streaming and yes
for streaming, the FD don't add much value. And I agree, keep-alive is probably  better for
that case. I'm merely disagreeing that for that case the FD removes so much value that it's
'non-sensicle' (given it's already there that is).

Anyway, in case that wasn't clear I'm not opposed to your proposition so please feel free
to submit a patch.

> Failure detector downs should not break streams
> -----------------------------------------------
>                 Key: CASSANDRA-3569
>                 URL:
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Peter Schuller
>            Assignee: Peter Schuller
> CASSANDRA-2433 introduced this behavior just to get repairs to don't sit there waiting
forever. In my opinion the correct fix to that problem is to use TCP keep alive. Unfortunately
the TCP keep alive period is insanely high by default on a modern Linux, so just doing that
is not entirely good either.
> But using the failure detector seems non-sensicle to me. We have a communication method
which is the TCP transport, that we know is used for long-running processes that you don't
want to incorrectly be killed for no good reason, and we are using a failure detector tuned
to detecting when not to send real-time sensitive request to nodes in order to actively kill
a working connection.
> So, rather than add complexity with protocol based ping/pongs and such, I propose that
we simply just use TCP keep alive for streaming connections and instruct operators of production
clusters to tweak net.ipv4.tcp_keepalive_{probes,intvl} as appropriate (or whatever equivalent
on their OS).
> I can submit the patch. Awaiting opinions.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message