incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Schuller <>
Subject Re: Error when bringing up nodes during failure testing
Date Tue, 08 Mar 2011 17:57:56 GMT
> 2) When I brought 2 nodes down (out of 3), I was able to start one node
> (with 66 % load below) even though auto_bootstrap is set to true. Shouldn't
> it have failed for the same reason?

This is a good point/question. As far as I can tell, a node being
bootstrapped would need to receive data from a sufficient number of
replicas to satisfy the maximum consistently level that the
application(s) use, in order to avoid the potential for violating the
consistency requirement expected by clients. Not knowing what the
application expects, that would imply a quorum of nodes.

I just checked the code, and my reading (untested) is that the intent
is to receive data from all nodes responsible for the part of the ring
that is being taken over. Meaning, it satisfies the above requirement.

However, that reading is inconsistent with your test which suggests
you were able to bootstrap with two nodes missing out of three.

Is your nodetool output from the new node or the pre-existing online
node? It only lists two nodes, rather than 3 or 4 (with some being
Down). If the only remaining node doesn't know about the other two
that are down, that may explain it.

I may be mis-reading the code because it's suddenly unclear to me how
this is supposed to work with respect to nodes being down (supposing
it's truly down, forever, and needs to be replaced).


/ Peter Schuller

View raw message