zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Hunt <ph...@apache.org>
Subject Re: Is ZooKeeper the right tool for our needs?
Date Mon, 04 Jun 2012 22:33:47 GMT
This paper from Usenix might shed some light as well, esp around how
things work and why we do what we do (i.e. majority quorums):

Good Luck,


On Mon, Jun 4, 2012 at 3:23 PM, Camille Fournier <camille@apache.org> wrote:
>> However, the
>> ZooKeeper client connects to a ZooKeeper server at random. This is another
>> indication that ZooKeeper might be the wrong tool to use.
> You seed the client with the list of servers to connect to. If you
> only seed it with one server, it will only connect to one server even
> if there are others in the quorum, so this should not be an issue.
> As for your questions, you're definitely pushing the envelope on the
> standard deployment model but ZK is probably the right tool for you to
> use if you care about correctness in your distributed locking. Using
> ZK for distributed locking is one of its key uses. Embedding it inside
> an application is not common but not unheard of. The danger with
> embedding ZK is that you now have another system that can cause ZK to
> perform poorly due to GC, cpu contention, etc. Is there some reason
> you're concerned with running ZK as a separate VM? It won't be highly
> available, but it sounds like running a standalone ZK on a separate
> server than your two system servers might be the best choice for you
> right now. I would probably recommend this over trying to run an
> embedded quorum.
> C
> On Mon, Jun 4, 2012 at 5:44 PM, David Nickerson <
> davidnickerson4mailinglists@gmail.com> wrote:
>> I work for a small software company and I have been tasked with researching
>> a distributed lock manager for us to use. I would appreciate any feedback
>> on whether or not ZooKeeper is the right choice.
>> Our software solution utilizes multiple servers, but only the Java
>> application called "Administrator" needs to acquire locks. The server that
>> Administrator runs on is called the System Server, which presents a web
>> interface to potentially thousands of users, and is highly multi-threaded.
>> So far, we have used only one System Server, and we have needed
>> locks only among threads in one application. However, we are looking to
>> scale the number of System Servers, and therefore need a distributed lock
>> manager to be shared among them.
>> We don't want to set up another server just for ZooKeeper, and we don't
>> want the System Server to run another instance of the JVM just for
>> ZooKeeper, so we have decided to embed the ZooKeeper server in our
>> Administrator application.
>> If the System Server goes down for whatever reason, we no longer have a
>> need for locks, so it's not a problem if ZooKeeper goes down too. However,
>> if we have two System Servers, we would like to be able to fail over to the
>> second server. However, ZooKeeper requires that a majority of the ZooKeeper
>> servers be up in order to run. This requirement seems strange and
>> worrisome. For the time being, we can just run an additional ZooKeeper
>> server on another server to give ZooKeeper the redundancy it needs.
>> Another issue is that we would like to have fast locks and reduced network
>> traffic by being able to read from the local ZooKeeper server. However, the
>> ZooKeeper client connects to a ZooKeeper server at random. This is another
>> indication that ZooKeeper might be the wrong tool to use.
>> An important feature that we need is deadlock detection. I had to write
>> this myself, but I was able to do so using the interface that ZooKeeper
>> provides.
>> We also need to be able to administrate the ZooKeeper server from our
>> program. For example, if the ZooKeeper server has an error or we need to
>> shut down the System Server, then the program may need to shutdown the
>> ZooKeeper server or restart it. Some code to do this might be:
>>    // To start the server
>>    QuorumPeerMain main;
>>    try {
>>      QuorumPeerConfig config = new QuorumPeerConfig();
>>      config.parse("ZooKeeper files\\zoo.cfg");
>>      main = new QuorumPeerMain();
>>      main.runFromConfig(config);
>>    } catch (ConfigException e) {
>>      e.printStackTrace();
>>    } catch (IOException e) {
>>      e.printStackTrace();
>>    }
>>    // To shutdown the server
>>    main.quorumPeer.shutdown();
>> However, "main.quorumPeer" is protected. There is very little documentation
>> on the Internet about embedding a ZooKeeper server. In fact, much of the
>> server interface seems to be written with the mindset that the ZooKeeper
>> server wouldn't be embedded in an application.
>> So, in conclusion, I have two important questions. First, will ZooKeeper
>> work for us, or are we trying to use it in a way that it's not designed
>> for? And second, if ZooKeeper is a good choice, then is our idea of
>> embedding the ZooKeeper server the right thing to do?
>> Thanks for reading this, and I appreciate any thoughts or ideas.

View raw message