zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Camille Fournier <cami...@apache.org>
Subject Re: Is ZooKeeper the right tool for our needs?
Date Mon, 04 Jun 2012 22:23:38 GMT
> However, the
> ZooKeeper client connects to a ZooKeeper server at random. This is another
> indication that ZooKeeper might be the wrong tool to use.

You seed the client with the list of servers to connect to. If you
only seed it with one server, it will only connect to one server even
if there are others in the quorum, so this should not be an issue.

As for your questions, you're definitely pushing the envelope on the
standard deployment model but ZK is probably the right tool for you to
use if you care about correctness in your distributed locking. Using
ZK for distributed locking is one of its key uses. Embedding it inside
an application is not common but not unheard of. The danger with
embedding ZK is that you now have another system that can cause ZK to
perform poorly due to GC, cpu contention, etc. Is there some reason
you're concerned with running ZK as a separate VM? It won't be highly
available, but it sounds like running a standalone ZK on a separate
server than your two system servers might be the best choice for you
right now. I would probably recommend this over trying to run an
embedded quorum.


On Mon, Jun 4, 2012 at 5:44 PM, David Nickerson <
davidnickerson4mailinglists@gmail.com> wrote:

> I work for a small software company and I have been tasked with researching
> a distributed lock manager for us to use. I would appreciate any feedback
> on whether or not ZooKeeper is the right choice.
> Our software solution utilizes multiple servers, but only the Java
> application called "Administrator" needs to acquire locks. The server that
> Administrator runs on is called the System Server, which presents a web
> interface to potentially thousands of users, and is highly multi-threaded.
> So far, we have used only one System Server, and we have needed
> locks only among threads in one application. However, we are looking to
> scale the number of System Servers, and therefore need a distributed lock
> manager to be shared among them.
> We don't want to set up another server just for ZooKeeper, and we don't
> want the System Server to run another instance of the JVM just for
> ZooKeeper, so we have decided to embed the ZooKeeper server in our
> Administrator application.
> If the System Server goes down for whatever reason, we no longer have a
> need for locks, so it's not a problem if ZooKeeper goes down too. However,
> if we have two System Servers, we would like to be able to fail over to the
> second server. However, ZooKeeper requires that a majority of the ZooKeeper
> servers be up in order to run. This requirement seems strange and
> worrisome. For the time being, we can just run an additional ZooKeeper
> server on another server to give ZooKeeper the redundancy it needs.
> Another issue is that we would like to have fast locks and reduced network
> traffic by being able to read from the local ZooKeeper server. However, the
> ZooKeeper client connects to a ZooKeeper server at random. This is another
> indication that ZooKeeper might be the wrong tool to use.
> An important feature that we need is deadlock detection. I had to write
> this myself, but I was able to do so using the interface that ZooKeeper
> provides.
> We also need to be able to administrate the ZooKeeper server from our
> program. For example, if the ZooKeeper server has an error or we need to
> shut down the System Server, then the program may need to shutdown the
> ZooKeeper server or restart it. Some code to do this might be:
>    // To start the server
>    QuorumPeerMain main;
>    try {
>      QuorumPeerConfig config = new QuorumPeerConfig();
>      config.parse("ZooKeeper files\\zoo.cfg");
>      main = new QuorumPeerMain();
>      main.runFromConfig(config);
>    } catch (ConfigException e) {
>      e.printStackTrace();
>    } catch (IOException e) {
>      e.printStackTrace();
>    }
>    // To shutdown the server
>    main.quorumPeer.shutdown();
> However, "main.quorumPeer" is protected. There is very little documentation
> on the Internet about embedding a ZooKeeper server. In fact, much of the
> server interface seems to be written with the mindset that the ZooKeeper
> server wouldn't be embedded in an application.
> So, in conclusion, I have two important questions. First, will ZooKeeper
> work for us, or are we trying to use it in a way that it's not designed
> for? And second, if ZooKeeper is a good choice, then is our idea of
> embedding the ZooKeeper server the right thing to do?
> Thanks for reading this, and I appreciate any thoughts or ideas.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message