cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kevin Burton <>
Subject Re: All writes fail with ONE consistency level when adding second node to cluster?
Date Wed, 23 Jul 2014 03:46:04 GMT
Yeah.. that's fascinating … so now I get something that's even worse:

"Cassandra timeout during write query at consistency ANY (2 replica were
required but only 1 acknowledged the write)"

… the issue is that the new cassandra node has all its ports closed.

Only the storage port is open.

So obviously writes are going to fail to it.

… is this by design?  Perhaps it's not going to open the ports until the
node joins the ring?  It's currently "joining" …

so… basically, my entire cluster is offline during this join?

I assume this is either a bug or some weird state base on growing from 1-2

frustrating :-(

On Tue, Jul 22, 2014 at 8:13 PM, graham sanderson <> wrote:

> Incorrect, ONE does not refer to the number of “other" nodes, it just
> refers to the number of nodes. so ONE under normal circumstances would only
> require one node to acknowledge the write.
> The confusing error message you are getting is related to
>… Kevin you are
> correct in that normally that error message would make no sense.
> I don’t have much experience adding/removing nodes, but I think what is
> happening is that your new node is in the middle of taken over ownership of
> a token range - while that happens C* is trying to write to both the old
> owner (your original node), AND (hence the 2 not 1 in the error message)
> the new owner (the new node) so that once the bootstrapping of the new node
> is complete, it is immediately safe to delete the (no longer owned data)
> from the old node. For whatever reason the write to the new node is timing
> out, causing the exception, and the error message is exposing the “2” which
> happens to be how many C* thinks it is waiting for at the time (i.e. how
> many it should be waiting for based on the consistency level (1) plus this
> extra node).
> On Jul 22, 2014, at 9:46 PM, Andrew <> wrote:
> ONE means write to one replica (in addition to the original).  If you want
> to write to any of them, use ANY.  Is that the right understanding?
> Andrew
> On July 22, 2014 at 7:43:43 PM, Kevin Burton ( wrote:
> I'm super confused by this.. and disturbed that this was my failure
> scenario :-(
> I had one cassandra node for the alpha of my app… and now we're moving
> into beta… which means three replicas.
> So I added the second node… but my app immediately broke with:
> ""Cassandra timeout during write query at consistency ONE (2 replica were
> required but only 1 acknowledged the write)""
> … but that makes no sense… if I'm at ONE and I have one acknowledged
> write, why does it matter that the second one hasn't ack'd yet…
> ?
> --
> Founder/CEO <>
> Location: *San Francisco, CA*
> blog:
> … or check out my Google+ profile
> <>
> <>


Location: *San Francisco, CA*
… or check out my Google+ profile

View raw message