Віталій Тимчишин:
> Hello.
>
> I am trying to implement distributed executor service over a zookeeper and
> encountered a problem I am not sure how to deal with.
> <SNIP />
Hi,
I've the impression, you're doing it too complicate. Why not have a znode
jobs/ and put all your jobs as persistent znodes in there.
So your job_queue could look like:
jobs/
job1
job2
job3
A consumer can then lock a job by putting an ephemeral node in one of the job
znodes:
jobs/
job1
job2/
.lock
job3
Other consumers see that somebody is already working on job2. If the consumer
should die, the ephemeral lock will vanish and another consumer could take
over.
You should not publish results in ZK but use some other communication method.
ZK is really not meant as a messaging solution.
Maybe you'd be better served with a standard message queue (e.g. Apache
activeMQ, rabbitMQ) and use ZK only for coordination? If you have really high
message load, you could have a look at Hedwig (google://zookeeper hedwig).
Best regards,
Thomas Koch, http://www.koch.ro
|