zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Scott Lindner <scott.a.lind...@gmail.com>
Subject Possibility / consequences of having multiple elected leaders
Date Wed, 07 Mar 2012 17:48:46 GMT
In debugging our problem where we had a zookeeper cluster failure (separate
thread) we ran across something that might have happened that could have
caused one of our servers to be quite a bit behind the other two.  We are
running a cluster of 3 zookeeper servers in our development cluster on
Windows.  These are not running as services and are just started from the
command prompt.  Because of this, it's possible that one of the servers had
their command output frozen by someone clicking / marking it.  We saw this
happen accidentally while debugging and the end result is obviously that
all requests to that server back up until they either time out or the
command prompt is unfrozen.

This got us to wondering what would happen if the elected leader were
"frozen" in this manner?  There's no guarantees where in the code it would
be hung to know for certain what would happen when it left this state, but
could there be any problems where the "frozen" server would come out of
this state still thinking it was the leader (since it was stuck) when in
fact another server had been elected in the meantime?  I would imagine this
should resolve itself fairly quickly but is there still a possibility that
this could lead to bad behavior?  Typically if a server fails I would
imagine the zookeeper instance would die or lose leadership because of an
event (failed connection, etc) but this seems slightly different since the
code would be blocked in a random state.

This seems to be more of a Windows issue given how its command prompts work
vs. other OS and we're going to avoid this by either installing a service
that is responsible for starting the zookeeper servers or piping the output
to a file where we can tail the output.

Thanks,
-Scott

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message