zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Hunt <ph...@apache.org>
Subject Re: Question on maintaining leader/membership status in zookeeper
Date Fri, 30 Apr 2010 20:45:11 GMT
It really depends on the type of leader election you have implemented. 
You can do simple leader election such as we describe in the recipes, in 
that case clients won't notice the leader changed unless they are 
connected. You could implement some more complex protocol, but typically 
the recipe we have is sufficient.

In my experience what you should do also depends on whether you have an 
"active" leader or a passive one.

A passive leader is one that is connected to by other hosts and thereby 
performs some action, an active leader is one that initiates actions.

Typically in the passive case if the leader gets disconnected from the 
server you just want to wait until you get reconnected to determine if 
you are still the leader. (hosts just won't connect to you if zk cluster 
notifies them that you are no longer the leader)

However in the active case while you are disconnected from the cluster 
someone else may have become the leader, in that case you should stop 
initiating actions (typically). You may want to stop acting as the 
leader as soon as you become disconnected from the ensemble because you 
have no way to know if/when someone else was elected as the leader.

Patrick

On 04/30/2010 01:26 PM, Lei Gao wrote:
> Hi Henry,
>
> I am not talking about the leader election within zookeeper cluster. I guess
> I didn't make the discussion context clear. In my case, I run a cluster that
> uses zookeeper for doing the leader election. Yes, nodes in my cluster are
> the clients of zookeeper.  Those nodes depend on zookeeper to elect a new
> leader and figure out what the current leader is. So if the zookeeper (think
> of it as a stand-alone entity) becomes unavailabe in the way I've described
> earlier, how can I handle such situation so my cluster can still function
> while a majority of nodes still connect to each other (but not to the
> zookeeper)?
>
> Thanks,
>
> Lei
>
>
> On 4/30/10 1:10 PM, "Henry Robinson"<henry@cloudera.com>  wrote:
>
>> Hi Lei -
>>
>> The 'user cluster' (by which I think you mean the set of clients of
>> ZooKeeper?) plays no part in leader election. If a majority of ZooKeeper
>> server nodes can talk to each other, a new leader can be elected. Clients of
>> the minority server partition will be disconnected - if they too cannot
>> reach the majority partition then they will not be able to reconnect.
>>
>> Hope this helps,
>> Henry
>>
>> On 30 April 2010 12:45, Lei Gao<lgao@linkedin.com>  wrote:
>>
>>> Hi Ted,
>>>
>>> I 100% agree with what you said. But my question is more about what if my
>>> zookeeper service cluster is partitioned from a majority of nodes in my USER
>>> CLUSTER.  In this case, the majority nodes in one network partition can¹t
>>> select a new leader because zookeeper is out of reach.
>>>
>>> Another example will be that if there is an asymmetric network failure
>>> where a majority of nodes from the USER CLUSTER can¹t reach the leader while
>>> the zookeeper still can. How does zookeeper handle such situation?
>>>
>>> Thanks,
>>>
>>> Lei
>>>
>>> On 4/30/10 12:24 PM, "Ted Dunning"<ted.dunning@gmail.com>  wrote:
>>>
>>> There are a variety of situations that can trigger a new leader election
>>> and a few that can cause the cluster to be unable to elect a new leader.
>>>   Isolation of just the leader is one of the situations that will cause a new
>>> leader election.  Isolation of nodes into groups smaller than the quorum
>>> will result in the cluster freezing.
>>>
>>> On Fri, Apr 30, 2010 at 11:56 AM, Lei Gao<lgao@linkedin.com>  wrote:
>>> Hi,
>>>
>>> I have a general question on how zookeeper can maintain its view of the
>>> user cluster (that zookeeper manages) that is consistent with the nodes in
>>> the user cluster. In other words, when zookeeper considers the current
>>> leader is unavailable, does it really guarantee that a majority of nodes in
>>> the user cluster can¹t reach the current leader? The same question applies
>>> to the membership service as well. Because the zookeeper can be partitioned
>>> from a majority of the nodes in the user cluster. How does the zookeeper
>>> handle situations like this?
>>>
>>> Thanks,
>>>
>>> Lei
>>>
>>>
>>>
>>
>

Mime
View raw message