cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Ballet <jbal...@edgelab.ch>
Subject How seed nodes are working and how to upgrade/replace them?
Date Mon, 07 Jan 2019 14:37:33 GMT
Hi,

I'm trying to understand how seed nodes are working, when and how do they
play a part in a Cassandra cluster, and how they should be managed and
propagated to other nodes.

I have a cluster of 6 Cassandra nodes (let's call them #1 to #6), on which
node #1 and #2 are seeds. All the configuration files of all the Cassandra
nodes are currently configured with:

```
seed_provider:
- class_name: org.apache.cassandra.locator.SimpleSeedProvider
  parameters:
  - seeds: 'IP #1,IP #2'
```

We are using a service discovery tool (Consul) which automatically
registers new Cassandra nodes with its dedicated health-check and are able
to generate new configuration based on the content of the service discovery
status (with Consul-Template).


I'm working on how we could improve the upgrades of our servers and how to
replace them completely (new instance with a new IP address).
What I would like to do is to replace the machines holding our current
seeds (#1 and #2 at the moment) in a rolling upgrade fashion, on a regular
basis:

* Is it possible to "promote" any non-seed node as a seed node?

* Is it possible to "promote" a new seed node without having to restart all
the nodes?
  In essence, in my example that would be:

  - decide that #2 and #3 will be the new seed nodes
  - update all the configuration files of all the nodes to write the IP
addresses of #2 and #3
  - DON'T restart any node - the new seed configuration will be picked up
only if the Cassandra process restarts

* If I can manage to sort my Cassandra nodes by their age, could it be a
strategy to have the seeds set to the 2 oldest nodes in the cluster? (This
implies these nodes would change as the cluster's nodes get
upgraded/replaced).


I also have some more general questions about seed nodes and how they work:

* I understand that seed nodes are used when a node starts and needs to
discover the rest of the cluster's nodes. Once the node has joined and the
cluster is stable, are seed nodes still playing a role in day to day
operations?

* The documentation says multiple times that not all nodes should be seed
nodes, but I didn't really find any place about the consequences it has to
have "too many" seed nodes. Also, relatively to the questions I asked
above, is there any downsides of having changing seed nodes in a cluster?
(with the exact same, at some point I define #1 and #2 to be seeds, then
later #4 and #5, etc.)


Thanks for helping me to understand better how seeds are working!

 Jonathan

Mime
View raw message