zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Koch <tho...@koch.ro>
Subject Re: ExecutorService over a zookeeper
Date Sun, 13 Feb 2011 16:10:20 GMT
Віталій Тимчишин:
> Hello.
> 
> I am trying to implement distributed executor service over a zookeeper and
> encountered a problem I am not sure how to deal with.
> <SNIP />
Hi,

I've the impression, you're doing it too complicate. Why not have a znode 
jobs/ and put all your jobs as persistent znodes in there.
So your job_queue could look like:

jobs/
  job1
  job2
  job3

A consumer can then lock a job by putting an ephemeral node in one of the job 
znodes:

jobs/
  job1
  job2/
    .lock
  job3

Other consumers see that somebody is already working on job2. If the consumer 
should die, the ephemeral lock will vanish and another consumer could take 
over.
You should not publish results in ZK but use some other communication method. 
ZK is really not meant as a messaging solution.
Maybe you'd be better served with a standard message queue (e.g. Apache 
activeMQ, rabbitMQ) and use ZK only for coordination? If you have really high 
message load, you could have a look at Hedwig (google://zookeeper hedwig).

Best regards,

Thomas Koch, http://www.koch.ro

Mime
View raw message