zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Samuel Rash <r...@fb.com>
Subject Re: ephemeral node problem
Date Thu, 17 Feb 2011 22:12:47 GMT

this is what we saw in our zk logs (CancelledKeyException)

Sam Rash

On 2/17/11 1:29 PM, "Mahadev Konar" <mahadev@apache.org> wrote:

>You probably ran into
>If yes, 3.3.3 has the fix! It is due to be released next week.
>On Thu, Feb 17, 2011 at 1:08 PM, Samuel Rash <rash@fb.com> wrote:
>> Hi,
>> We are running zookeeper 3.3.2 and have seen what appears to be a
>>problem with ephemeral nodes.  We create about 2000 persistent nodes
>>(leaves) in a hierarchy.  Under each of these, we run a leader
>> election with ephemeral nodes (~40).  This results in about 80,000
>>total ephemeral nodes.  During restart of our system, the leader
>>elections can churn a bit as hosts remove themselves, electing new
>>leaders, which then themselves may withdraw from the election.  In one
>>such restart, we saw an election get 'stuck'.  Upon investigating, one
>>node had it's session expired (indicated in the zk logs), but one of its
>>ephemeral nodes was still left.  We took down the processes holdilng
>>ephemeral nodes and this node remained.
>> Are there any known bugs in zookeeper that might result in this?  it
>>does not appear to happen under our normal laod.
>> thx,
>> -sr
>> Sam Rash
>> rash@fb.com

View raw message