zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Schilli...@perlmeister.com>
Subject Re: Recommended ZooKeeper backup procedures/scripts
Date Thu, 20 Oct 2011 18:35:55 GMT
On Wed, 19 Oct 2011, Patrick Hunt wrote:

> Rsync is probably sufficient.

Interesting ... could one use rsync for BCP purposes?

So if I have two farms of zookeepers, could I rsync data from one farm
to the other, shut down the first and start up the other?

I presume that you guys recommend using ZooKeeper's quorum model
for BCP purposes, but how would you typically distribute ZooKeeper instances 
between different colos? Say, you have N colos, how many ZooKeeper instances
would you need to form a quorum, even if 1, or 2, or N-1 colos fail?

For example, if you have two colos, and 2 ZooKeeper instances in the
first and 3 ZooKeeper instances in the other, you'd have a total of 
5 instances which is a valid quorum. But as soon as the colo with 3
instances goes down, you're left with 2 instances which is an even
number. What do you guys recommend?

-- Mike

Mike Schilli
> The cleanup script will remove excess (ie old) snaps/logs, you could
> run that, then rsync the directory. Or rsync then cleanup (depending
> on how concerned you are that the N most recent snaps/logs might be
> corrupt - but then you could use something like LogFormatter to verify
> the log before cleanup, etc....)
> IIRC there was also the idea of using an observer as a backup - the
> observer would mirror the content of the serving cluster. See previous
> discussion here: http://bit.ly/oz26gN
> Patrick
> On Wed, Oct 19, 2011 at 1:38 PM, Mike Schilli <mschilli@yahoo-inc.com> wrote:
>> Is there a defined process on how to backup and restore ZooKeeper data?
>> The administrator's guide talks about the data directory, which
>> I suppose I could comb through and figure out what I need to back up
>> and what's superfluous.
>> Just checking if there's a script in the ZooKeeper distribution that
>> does backups and restores.
>> --
>> -- Mike
>> Mike Schilli
>> mschilli@yahoo-inc.com

View raw message