zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Newman <st...@scalyr.com>
Subject Re: Adding and removing replicas?
Date Thu, 20 Oct 2016 16:26:02 GMT
Thanks for the pointer. This looks like a nice improvement.

If I'm reading http://zookeeper.apache.org/releases.html correctly, this
feature is only available in alpha release. In the near term, I need a
procedure I can follow for a stable release. Is there any documentation
regarding best practices for adding / removing replicas prior to 3.5.0? The
3.5.2 documentation you linked to is somewhat alarming regarding prior
releases:

"Prior to the 3.5.0 release, the membership and all other configuration
parameters of Zookeeper were static - loaded during boot and immutable at
runtime. Operators resorted to ''rolling restarts'' - a manually intensive
and error-prone method of changing the configuration *that has caused data
loss and inconsistency in production*."

Thanks,
Steve

On Thu, Oct 20, 2016 at 8:03 AM, Rakesh Radhakrishnan <rakeshr@apache.org>
wrote:

> Hi Steve,
>
> I'd suggest you to look at ZooKeeper-3.5.2 latest version and use dynamic
> reconfig feature. This will help to resize(add/remove zk server) your
> cluster without restarting entire cluster.
>
> Please refer the following links to understand more about the dynamic
> reconfig feature:-
> https://zookeeper.apache.org/doc/r3.5.2-alpha/zookeeperReconfig.html
> http://www.slideshare.net/Hadoop_Summit/dynamic-
> reconfiguration-of-zookeeper
>
> Regards,
> Rakesh
>
> On Thu, Oct 20, 2016 at 3:19 AM, Steve Newman <steve@scalyr.com> wrote:
>
>> Apologies for a basic question, but I've been researching and haven't been
>> able to find the answer online.
>>
>> What is the best way to add or remove replicas from a running ZooKeeper
>> cluster, with minimal downtime? To add a replica, the naive answer would
>> seem to be:
>>
>> 1. Prepare the new replica(s), i.e. install ZooKeeper and set up the
>> configuration files.
>> 2. Edit the configuration for all replicas (new and existing) to list the
>> new replicas.
>> 3. Restart all replicas. (Simultaneously? Or gradually, one at a time?)
>>
>> Is this the best way to do it? Step 3 seems scary in a production cluster.
>> Also, will the new replicas smoothly pick up the existing data, or is it
>> better to seed them with a snapshot somehow?
>>
>> Similarly, the naive answer for removing a replica would seem to be:
>>
>> 1. Halt the ZooKeeper process.
>> 2. Edit the configuration for all other replicas to remove the replica
>> that's going away.
>> 3. Restart all remaining replicas (one at a time?).
>>
>> Again, is this the best approach?
>>
>> Thanks,
>> Steve
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message