cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Furlong <mfurl...@ancestry.com>
Subject RE: Node failure
Date Fri, 06 Oct 2017 17:42:33 GMT
Thanks for the detail. I’ll have to remove and then add one back in. It’s my consistency
levels that may bite me in the interim.

Thanks
Mark
801-705-7115 office

From: Jeff Jirsa [mailto:jjirsa@gmail.com]
Sent: Friday, October 6, 2017 11:29 AM
To: cassandra <user@cassandra.apache.org>
Subject: Re: Node failure

There's a lot to talk about here, what's your exact question?


- You can either remove it from the cluster or replace it. You typically remove it if it'll
never be replaced, but in RF=3 with 3 nodes, you probably need to replace it. To replace,
you'll start a new server with -Dcassandra.replace_address=a.b.c.d ( http://cassandra.apache.org/doc/latest/operating/topo_changes.html#replacing-a-dead-node
) , and it'll stream data from the neighbors and eventually replace the dead node in the ring
(the dead node will be removed from 'nodetool status', the new node will be there instead).

- If you're not going to replace it, things get a bit more complex - you'll do some combination
of repair, 'nodetool removenode' or 'nodetool assassinate', and ALTERing the keyspace to set
RF=2. The order matters, and so does the consistency level you use for reads/writes (so we
can tell you whether or not you're likely to lose data in this process), so I'm not giving
step-by-steps here because it's not very straight forward and there are a lot of caveats.




On Fri, Oct 6, 2017 at 10:20 AM, Mark Furlong <mfurlong@ancestry.com<mailto:mfurlong@ancestry.com>>
wrote:
What happens when I have a 3 node cluster with RF 3 and a node fails that needs to be removed?

Mark Furlong

Sr. Database Administrator

mfurlong@ancestry.com<mailto:mfurlong@ancestry.com>
M: 801-859-7427<tel:(801)%20859-7427>
O: 801-705-7115<tel:(801)%20705-7115>
1300 W Traverse Pkwy<https://maps.google.com/?q=1300+W+Traverse+Pkwy%0D+Lehi,+UT+84043&entry=gmail&source=g>
Lehi, UT 84043<https://maps.google.com/?q=1300+W+Traverse+Pkwy%0D+Lehi,+UT+84043&entry=gmail&source=g>





​[http://c.mfcreative.com/mars/email/shared-icon/sig-logo.gif]




Mime
View raw message