cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Dejanovski <a...@thelastpickle.com>
Subject Re: Rebuilding a node without clients hitting it
Date Tue, 06 Aug 2019 10:24:02 GMT
Hi Cyril,

it will depend on the load balancing policy that is used in the client code.

If you're only accessing DC1, with the node being rebuilt living in DC2,
then you need your clients to be using the DCAwareRoundRobinPolicy to
restrict connections to DC1 and avoid all kind of queries hitting DC2.
If clients are accessing both datacenters, and you're not using the
TokenAwarePolicy, even with LOCAL_ONE, the coordinator could pick the node
being rebuilt to process the query.

If you're not spinning up a new datacenter in an existing cluster,
rebuilding a node is not the best way to achieve this without compromising
consistency.
The node should be replaced, which will make it bootstrap safely (he can
replace himself, using the
"-Dcassandra.replace_address_first_boot=<address_of_replaced_node>" flag.
Bootstrap lets the node stream the data it needs faster than repair would,
while keeping it out of read requests.
The procedure is to stop Cassandra, wipe data, commit log and saved caches,
and then restart it with the JVM flag set in cassandra-env.sh. The node
will appear as joining or down while bootstrapping (it depends if it
replaces itself or another node, can't remember the specifics).
If it shows up as down, it will rely on hints to get the writes. If it
shows as joining, it will get the writes while streaming is ongoing.

Cheers,

-----------------
Alexander Dejanovski
France
@alexanderdeja

Consultant
Apache Cassandra Consulting
http://www.thelastpickle.com


On Tue, Aug 6, 2019 at 12:03 PM Cyril Scetbon <cyril.scetbon@free.fr> wrote:

> Can you elaborate on that ? We use GPFS
> without cassandra-topology.properties.
> —
> Cyril Scetbon
>
> On Aug 5, 2019, at 11:23 PM, Jeff Jirsa <jjirsa@gmail.com> wrote:
>
> some snitch trickery (setting the badness for the rebuilding host) via jmx
>
>
>

Mime
View raw message