zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jordan Zimmerman <jor...@jordanzimmerman.com>
Subject Re: Distributed Task Queue with client node failovers
Date Wed, 02 Sep 2015 14:51:56 GMT
Please have a look at Nirmata Workflow (which I wrote). Given your description, it does what
you want. Even if you don’t use it, you can review it for an example of how to do what you
want: http://nirmataoss.github.io/workflow/

-Jordan


On September 2, 2015 at 5:14:47 AM, Prabhjot Bharaj (prabhbharaj@gmail.com) wrote:

Hi Adam,  

I had not gone through the contents of the link you provided  
But, after reading the Tech Note 4, I can say that my use case is not on  
that scale.  

Regards,  
Prabhjot  

On Wed, Sep 2, 2015 at 4:09 PM, Adam Milne-Smith <adam@milne-smith.co.uk>  
wrote:  

> Not an answer to your question but I just wanted to check you've read  
> https://cwiki.apache.org/confluence/display/CURATOR/TN4  
>  
> On 2 Sep 2015 09:47, Prabhjot Bharaj <prabhbharaj@gmail.com> wrote:  
> >  
> > Hello Folks,  
> >  
> > I am trying to design a distributed queue with Zookeeper. The use case is  
> > like this:-  
> >  
> > A set of 't' tasks to be executed by 'n' nodes in round-robin fashion.  
> But,  
> > if any node goes down, the other nodes should take the task.  
> >  
> > For this purpose, I'm reading about zookeeper Queue recipe. The Queue  
> > recipe relies on client watcher.  
> >  
> > Initially, all clients run their respective tasks. But, after one  
> > iteration, lets say one client node goes down.  
> > In this case, that client node itself is unavailable to get a Watch  
> Event,  
> > can the task be transferred to another node ?  
> >  
> > The worst case will be -  
> > All the client nodes, but one, are down. Can this one node still pick up  
> > all tasks sequentially and execute them ?  
> >  
> > Thanks,  
> > Prabhjot  
>  



--  
---------------------------------------------------------  
"There are only 10 types of people in the world: Those who understand  
binary, and those who don't"  

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message