zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jordan Zimmerman <jor...@jordanzimmerman.com>
Subject Re: multiple machines working on different task
Date Wed, 09 Sep 2015 20:26:42 GMT
Nirmata Workflow is a distributed scheduler. From what I can read, that’s what you’ve described.
Have I missed something?


On September 4, 2015 at 5:29:33 PM, Check Peck (comptechgeeky@gmail.com) wrote:

Thanks Jordan.

Can you briefly explain how I can leverage Nirmata Workflow here? I was going through this
example http://nirmataoss.github.io/workflow/example.html but I was not able to understand
how can I use it for my use case?

And if you have some other solutions as well which I can use here then let me know as well.

On Fri, Sep 4, 2015 at 3:20 PM, Jordan Zimmerman <jordan@jordanzimmerman.com> wrote:
There are many ways to accomplish this. However, I wrote something that may solve your problem.
Have a look at Nirmata Workflow: http://nirmataoss.github.io/workflow/

Even if you don’t use it directly, you can use the code as a sample of how to do what you


On September 4, 2015 at 3:18:23 PM, Check Peck (comptechgeeky@gmail.com) wrote:

I have a simple requirement as I am using Zookeeper with Curator. I have a
znode like this "/test/clients/" and some other application will create a
znode under "/test/clients/" with some data in it and those are each task
that my other three machines has to perform so it will be like this:-

This will be a znode "/test/clients/clientid_123" with some data in it,
let's say Hello World 1
This will be a znode "/test/clients/clientid_124" with some data in it.
let's say Hello World 2
This will be a znode "/test/clients/clientid_125" with some data in it.
let's say Hello World 3
This will be a znode "/test/clients/clientid_126" with some data in it.
let's say Hello World 4

Now I have three machines in which I will be running a service which will
use Curator library. I want all those three machine to pick up some
clientid task from "/test/clients/" znode.

For example : if machineA is working on "clientid_123", then I don't want
machineB or machineC working on "clientid_123". A single machine can work
on multiple clientid task but no two machines should pick same client id

How can I do this with zookeeper? I can have all three machines keep a
watch on "/test/clients/" znode and as soon as anything gets added up in
this znode then I don't know how to make sure only one is picking client id
task. I can use leadership but in leadership, only single machine will be
leader and that single machine will work on all the client id task and
that's what I dont want.

I want all the machines working on some client id task.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message