zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Nickerson <davidnickerson4mailingli...@gmail.com>
Subject Is ZooKeeper the right tool for our needs?
Date Mon, 04 Jun 2012 21:44:52 GMT
I work for a small software company and I have been tasked with researching
a distributed lock manager for us to use. I would appreciate any feedback
on whether or not ZooKeeper is the right choice.

Our software solution utilizes multiple servers, but only the Java
application called "Administrator" needs to acquire locks. The server that
Administrator runs on is called the System Server, which presents a web
interface to potentially thousands of users, and is highly multi-threaded.

So far, we have used only one System Server, and we have needed
locks only among threads in one application. However, we are looking to
scale the number of System Servers, and therefore need a distributed lock
manager to be shared among them.

We don't want to set up another server just for ZooKeeper, and we don't
want the System Server to run another instance of the JVM just for
ZooKeeper, so we have decided to embed the ZooKeeper server in our
Administrator application.

If the System Server goes down for whatever reason, we no longer have a
need for locks, so it's not a problem if ZooKeeper goes down too. However,
if we have two System Servers, we would like to be able to fail over to the
second server. However, ZooKeeper requires that a majority of the ZooKeeper
servers be up in order to run. This requirement seems strange and
worrisome. For the time being, we can just run an additional ZooKeeper
server on another server to give ZooKeeper the redundancy it needs.

Another issue is that we would like to have fast locks and reduced network
traffic by being able to read from the local ZooKeeper server. However, the
ZooKeeper client connects to a ZooKeeper server at random. This is another
indication that ZooKeeper might be the wrong tool to use.

An important feature that we need is deadlock detection. I had to write
this myself, but I was able to do so using the interface that ZooKeeper
provides.

We also need to be able to administrate the ZooKeeper server from our
program. For example, if the ZooKeeper server has an error or we need to
shut down the System Server, then the program may need to shutdown the
ZooKeeper server or restart it. Some code to do this might be:
    // To start the server
    QuorumPeerMain main;
    try {
      QuorumPeerConfig config = new QuorumPeerConfig();
      config.parse("ZooKeeper files\\zoo.cfg");
      main = new QuorumPeerMain();
      main.runFromConfig(config);
    } catch (ConfigException e) {
      e.printStackTrace();
    } catch (IOException e) {
      e.printStackTrace();
    }
    // To shutdown the server
    main.quorumPeer.shutdown();

However, "main.quorumPeer" is protected. There is very little documentation
on the Internet about embedding a ZooKeeper server. In fact, much of the
server interface seems to be written with the mindset that the ZooKeeper
server wouldn't be embedded in an application.

So, in conclusion, I have two important questions. First, will ZooKeeper
work for us, or are we trying to use it in a way that it's not designed
for? And second, if ZooKeeper is a good choice, then is our idea of
embedding the ZooKeeper server the right thing to do?

Thanks for reading this, and I appreciate any thoughts or ideas.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message