cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Martin Lansler <martin.lans...@gmail.com>
Subject Best practices when deploying & upgrading a cassandra cluster
Date Thu, 11 Aug 2011 14:30:11 GMT
(Note: This is a repost from another thread which did not have a
relevant subject, sorry for the spamming)

Hi Eldad / All,

On Wed, Aug 10, 2011 at 8:32 AM, Eldad Yamin <eldad87@gmail.com> wrote:
> Can you please explain how did you upgraded. something like step-by-step.
> Thanks!

I took the liberty of replying to the group as it would be interesting
to hear how other folks out there are doing it...

I'm *not* running a prod system, just a test system of three nodes on
my laptop. So it would be nice to hear about real setups. Here is my
test setup:

apache-cassandra -> apache-cassandra-0.8.3
apache-cassandra-0.8.2/
apache-cassandra-0.8.3/
node1/
node2/
node3/

All nodeX look like:
bin -> ../apache-cassandra/bin/
commitlog/
conf/
data/
interface -> ../apache-cassandra/interface/
lib -> ../apache-cassandra/lib/
saved_caches/

The 'conf' directory is copied into each node from the virgin
cassandra distribution. I then create a local GIT repo and add the
'conf' directory so I can track any configuration changes on a node.
Then relevant node specific configuration settings are set. The
'commitlog', 'data' and 'saved_caches' are created by cassandra and
must be configured in 'cassandra.yaml' for each node.

When I upgrade I do the following:

1.
Make a diff of the new conf files from the new version so that  get
new parameters etc... I use emacs ediff-mode.
2.
Remove the old "apache-cassandra" symlink and point it to the new cassandra dist
3.
In a rolling fashion stop one node, and then restart it... as the
symlink is changes it will then boot with the upgraded cassandra dist.
(remember to cd out & in of the bin/ dir otherwise you will still be
in the old directory).
(4).
Should something break... just re-create the old symlink and restart
the node (provided cassandra has not performed any non backwards
compatible changes to the db files, should be noted in the README)

That's pretty much it.

On a prod setup one would probably use a tool such as puppet
(www.puppetlabs.com/) to ease setting up on many nodes... But there
are many ways to do this, for instance pssh
(http://code.google.com/p/parallel-ssh/).

Regards,
-Martin

Mime
View raw message