zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Shraer <shra...@gmail.com>
Subject Re: Dynamic reconfiguration
Date Sat, 28 Jul 2012 01:42:35 GMT
Yes, this entry will be deleted. I don't like this either - if a new
follower reboots before added to the config it will not be able to boot up
without manual help from an admin. That's why I'm considering maybe to
remove the check that a participant must always initially be in its own
config, but for now its there.

Alex

On Fri, Jul 27, 2012 at 6:34 PM, Jared Cantwell <jared.cantwell@gmail.com>wrote:

> Sorry for the confusion in terminology, I was unfamiliar with the exact
> leader/follower semantics previously.
>
> So if all connected servers update their config file, does that mean that
> non-voting followers who aren't part of the new ensemble will lose the
> entry specific to them in their config file?  I can test this myself, but
> getting an inside perspective is very helpful.
>
> Thanks again for the help!
> Jared
>
>
> On Jul 27, 2012, at 6:55 PM, Alexander Shraer <shralex@gmail.com> wrote:
>
> Yes, any number of followers which are not in the configuration can just
> connect and listen in. This has always been the case, also in 3.4, I just
> made use of this for the purpose of adding members during reconfiguration.
> Moreover, in 3.4 there this bug ZOOKEEPER-1113<https://issues.apache.org/jira/browse/ZOOKEEPER-1113>
> where the leader actually counts the votes of anyone connected, regardless
> of config membership :) This is fixed in ZK-107, so they are really
> non-voting followers.
>
> >   I am assuming that's the case, and that it is a follower (and not
> > participant) by virtue of not being in the official configuration stored
> in
> > zookeeper itself.
>
> Follower and participant types of servers is not something that was
> defined in ZK-107. In ZooKeeper every follower/leader is a "participant".
> Its just that the votes of participants that are not in the configuration
> are not counted that's why we call them non-voting followers. BTW,
> obviously a non-voting follower can not become leader (like ZK-1113 this
> was also not enforced before ZK-107).
>
> > And a followup... does zookeeper only overwrite the dynamic
> > configuration file for nodes that are voting participants?  Such that if
> I
> > started a follower and then left it running through some
> > reconfigurations, its file would not get updated if it was never added
> as
> > part of those reconfigurations?
>
> No, as soon as it connects to the current leader, its dynamic config file
> is overwritten with the current configuration as part of the
> synchronization with the leader. Every time a new configuration is
> committed, all connected servers (voting, non-voting, observers) will
> update their dynamic config file, doesn't matter if they're in the config.
>
> Alex
>
> On Fri, Jul 27, 2012 at 5:35 PM, Jared Cantwell <jared.cantwell@gmail.com>wrote:
>
>> So does just having the server started and pointing to the existing
>> ensemble automatically make it a "non participating follower"?  In other
>> words, there is no need to inform the existing nodes that this new node is
>> joining as a follower?  And to extend that, there could be any number of
>> followers that are simply listening in on the event stream?  I am assuming
>> that's the case, and that it is a follower (and not participant) by virtue
>> of not being in the official configuration stored in zookeeper itself.
>>
>> On Fri, Jul 27, 2012 at 6:29 PM, Alexander Shraer <shralex@gmail.com>wrote:
>>
>>> there are just two supported types - participant and observer.
>>> (participant can act as either follower or leader).
>>>
>>> So you can either write participant or leave it unspecified (which means
>>> participant by default). Also, since the ip is the same for all your ports
>>> you don't have to write it twice.  All of these should work in the same way:
>>>
>>> server.5=10.10.5.17:2182:2183:participant;10.10.5.17:2181
>>> server.5=10.10.5.17:2182:2183:participant;2181 <http://10.10.5.17:2181/>
>>> server.5=10.10.5.17:2182:2183;10.10.5.17:2181
>>> server.5=10.10.5.17:2182:2183;2181 <http://10.10.5.17:2181/>
>>>
>>>
>>>
>>> On Fri, Jul 27, 2012 at 5:25 PM, Jared Cantwell <
>>> jared.cantwell@gmail.com> wrote:
>>>
>>>> Thanks Alex for the response.  Our current lines in the configuration
>>>> look like this:
>>>>
>>>> server.5=10.10.5.17:2182:2183:participant;10.10.5.17:2181
>>>>
>>>> For the new servers is it ok for their entry to have "participant"?  Or
>>>> should that be something different (e.g. "follower")?
>>>>
>>>> ~Jared
>>>>
>>>> On Fri, Jul 27, 2012 at 6:20 PM, Alexander Shraer <shralex@gmail.com>wrote:
>>>>
>>>>> Hi Jared,
>>>>>
>>>>> Thanks for experimenting with this feature.
>>>>>
>>>>> The idea is that new servers join as "non voting followers". Which
>>>>> means that they act as normal followers but the leader ignores their
votes
>>>>> since they are not part of the current configuration. The leader only
>>>>> counts their votes during the reconfiguration itself (to make sure a
quorum
>>>>> of the new config is ready before the new config can be
>>>>> committed/activated). Defining them as observers is not a good idea,
for
>>>>> example in your scenario if they were observers they wouldn't be able
to
>>>>> participate in the reconfiguration protocol (which is similar to the
>>>>> protocol for committing any other operation in which observers don't
>>>>> participate) and since we don't have a quorum of followers in the new
>>>>> config that can ack, reconfiguration would throw an exception (of
>>>>> KeeperException.NEWCONFIGNOQUORUM type).
>>>>> Of course if you intend them to be observers in the new config you can
>>>>> define them as observers since their votes are not needed during reconfig
>>>>> anyway.
>>>>>
>>>>> You're right, the new servers must be able to connect to the old
>>>>> quorum. At minimum, their file should contain the current leader, but
>>>>> you can also copy the current configuration file to the new members if
>>>>> you wish.
>>>>>
>>>>> In addition, you should add a line for the member itself, so that
>>>>> server F appears in F's config file (Its not important that the other
new
>>>>> servers appear in F's file, but it won't hurt either, so you can do a
union
>>>>> of old and new if you wish). The constructor of QuorumPeer checks that
the
>>>>> server itself is in the configuration its started with, otherwise its
not
>>>>> going to run. This check has always been there, but I'm thinking of
>>>>> possibly changing it in the future.
>>>>>
>>>>> As soon as F connects to the leader, its config file will be
>>>>> overwritten with the current config file as part of the synchronization
>>>>> process.
>>>>>
>>>>> Alex
>>>>>
>>>>>
>>>>> On Fri, Jul 27, 2012 at 10:06 AM, Jared Cantwell <
>>>>> jared.cantwell@gmail.com> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> We are testing integration with 3.5.0 and dynamic membership and
I
>>>>>> have a
>>>>>> question.  If I have a current set of servers in my ensemble
>>>>>> {A,B,C,D,E}
>>>>>> and I want to reconfigure the ensemble to {D,E,F,G,H}, how should
the
>>>>>> dynamic config file on servers F,G,H be configured on startup?
>>>>>>  Should they
>>>>>> have the old ensemble, the new ensemble, or the union of both
>>>>>> ensembles?
>>>>>>  It seems like these new servers need to  know about the old quorum,
>>>>>> but
>>>>>> since they aren't part of it yet its not clear to me how they should
>>>>>> be
>>>>>> configured.  Should there be an intermediate configuration with F,G,
>>>>>> and H
>>>>>> as simply Observers?
>>>>>>
>>>>>> I can't find much documentation on this so I want to make sure I
>>>>>> understand
>>>>>> things correctly.
>>>>>>
>>>>>> Thanks!
>>>>>> ~Jared
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message