cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Paul Lathrop (Commented) (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-4162) nodetool disablegossip does not prevent gossip delivery of writes via already-initiated hinted handoff
Date Wed, 18 Apr 2012 01:23:13 GMT


Paul Lathrop commented on CASSANDRA-4162:

If you are not going to actually "effectively mark the node dead" you shouldn't advertise
it as such in the nodetool documentation.

This definitely violates the principle of least surprise, in my opinion. At a bare minimum
the docs should be updated. However, it would be good to go the next step and actually support
the use case that production users actually encounter instead of dismissing it because you
can't think of a scenario where you'd use it.

Put another way:

"As an operator of a cassandra cluster, I want a reliable way to remove a node from the cluster
and disable traffic to it, so that I can diagnose problems with the node while keeping it
from participating in the cluster." No, iptables is not the correct answer to this use case.
> nodetool disablegossip does not prevent gossip delivery of writes via already-initiated
hinted handoff
> ------------------------------------------------------------------------------------------------------
>                 Key: CASSANDRA-4162
>                 URL:
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.0.9
>         Environment: reported on IRC, believe it was a linux environment, nick "rhone",
cassandra 1.0.8
>            Reporter: Robert Coli
>            Priority: Minor
>              Labels: gossip
> This ticket derives from #cassandra, aaron_morton and I assisted a user who had run "disablethrift"
and "disablegossip" and was confused as to why he was seeing writes to his node.
> Aaron and I went through a series of debugging questions, user verified that there was
traffic on the gossip port. His node was showing as down from the perspective of other nodes,
and nodetool also showed that gossip was not active.
> Aaron read the code and had the user turn debug logging on. The user saw Hinted Handoff
messages being delivered and Aaron confirmed in the code that a hinted handoff delivery session
only checks gossip state when it first starts. As a result, it will continue to deliver hints
and disregard gossip state on the target node.
> per nodetool docs
> "
> disablegossip          - Disable gossip (effectively marking the node dead)
> "
> I believe most people will be using disablegossip and disablethrift for operational reasons,
and propose that they do not expect HH delivery to continue, via gossip, when they have run

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message