cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anthony Grasso <>
Subject Re: How do you do automatic restacking of AWS instance for cassandra?
Date Sun, 28 May 2017 08:24:15 GMT
Hi Surbhi,

Please see my comment inline below.

On 28 May 2017 at 12:11, Jeff Jirsa <> wrote:

> On 2017-05-27 18:04 (-0700), Surbhi Gupta <>
> wrote:
> > Thanks a lot for all of your reply.
> > Our requirement is :
> > Our company releases AMI almost every month where they have some or the
> > other security packages.
> > So as per our security team we need to move our cassandra cluster to the
> > new AMI .
> > As this process happens every month, we would like to automate the
> process .
> > Few points to consider here:
> >
> > 1. We are using ephemeral drives to store cassandra data
> > 2. We are on dse 4.8.x
> >
> > So currently to do the process, we pinup a new nodes with new DC name and
> > join that DC, alter the keyspace, do rebuild  and later alter the
> keyspace
> > again to remove the old DC .
> >
> > But all of this process is manually done as of now.
> >
> > So i wanted to understand , on AWS, how do you do above kind of task
> > automatically ?
> At a previous employer, they used M4 class instances with data on a
> dedicated EBS volumes, so we could swap AMIs / stop / start / adjust
> instances without having to deal with this. This worked reasonably well for
> their scale (which was petabytes of data).

This is a really good option as it avoids streaming data to replace a node
which could potentially be quicker if dealing with large amounts of data on
each node.

> Other companies using ephemeral tend to be more willing to just terminate
> instances and replace them (-Dcassandra.replace_address). If you stop
> cassandra, then boot a replacement with 'replace_address' set, it'll take
> over for the stopped instance, including re-streaming all data (as best it
> can, subject to consistency level and repair status). This may be easier
> for you to script than switching your fleet to EBS, but it's not without
> risk.

A quick note if you do decide to go down this path. If you are using
Cassandra version 2.x.x and above, the cassandra.replace_address_first_boot
can also be used. This option works once when Cassandra is first started
and the replacement node inserted into the cluster. After that, the option
is ignored for all subsequent restarts, where as
cassandra.replace_address needs
to be removed from the ** file in order to restart the
node. Restart behaviour aside, both options operate in the same way to
replace a node in the cluster.

> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

View raw message