zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Dunning <ted.dunn...@gmail.com>
Subject Re: Task/Job distribution using ZooKeeper
Date Mon, 07 Mar 2011 16:43:14 GMT
Yes and no.

The worker may miss a notification.

But that isn't a problem because it shouldn't miss any data.

The idea is that only one notification will go to the worker.  If the server
changes the file again before the worker gets around to looking at the data,
no additional notification will be given to the worker.  The worker will set
a new notification in the same operation as it reads the data so it will get
both updates and a new notification will be set to tell it about any further
updates.

I think you design is a bit short on notifying the master of work that is
finished.  Also, I don't think you handle worker failure in a nice resilient
way.  Camille's suggestion of a single job queue can be extended to handle
both issues.

Good luck!

On Mon, Mar 7, 2011 at 2:41 AM, Sabyasachi Ruj <ruj.sabya@gmail.com> wrote:

> 3. In the example above, is there a chance that the worker may miss
> notification for job Y?
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message