cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shishir Kumar <shishirroy2...@gmail.com>
Subject Upgrade strategy for high number of nodes
Date Fri, 29 Nov 2019 15:21:53 GMT
Hi,

Need input on cassandra upgrade strategy for below:
1. We have Datacenter across 4 geography (multiple isolated deployments in
each DC).
2. Number of Cassandra nodes in each deployment is between 6 to 24
3. Data volume on each nodes between 150 to 400 GB
4. All production environment has DR set up
5. During upgrade we do not want downtime

We are planning to go for stack upgrade but upgradesstables is taking
approx. 5 hours per node (if data volume is approx 200 GB).
Options-
No downtime - As per recommendation (DataStax documentation) if we plan to
upgrade one node at time I.e. in sequence upgrade cycle for one environment
will take weeks, so DevOps concern.
Read Only (No downtime) - Route read only load to DR system. We have
resilience built up to take care of mutation scenarios. But incase it takes
more than say 3-4 hours, there will be long catch up exercise. Maintenance
cost seems too high due to unknowns
Downtime- Can upgrade all nodes in parallel as no live customers. This has
direct Customer impact, so need to convince on maintenance cost vs customer
impact.
Please suggest how other Organisation are solving this scenario (whom have
100+ nodes)

Regards
Shishir

Mime
View raw message