zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Віталій Тимчишин <tiv...@gmail.com>
Subject Re: ExecutorService over a zookeeper
Date Mon, 14 Feb 2011 08:35:47 GMT
13 лютого 2011 р. 18:10 Thomas Koch <thomas@koch.ro> написав:

> Віталій Тимчишин:
> > Hello.
> >
> > I am trying to implement distributed executor service over a zookeeper
> and
> > encountered a problem I am not sure how to deal with.
> > <SNIP />
> Hi,
>
> I've the impression, you're doing it too complicate. Why not have a znode
> jobs/ and put all your jobs as persistent znodes in there.
> So your job_queue could look like:
>
> jobs/
>  job1
>  job2
>  job3
>
> A consumer can then lock a job by putting an ephemeral node in one of the
> job
> znodes:
>
> jobs/
>  job1
>  job2/
>    .lock
>  job3
>
>
1)This won't handle producer death as job can't be EPHEMERAL
2)The job list will have locked jobs every consumer must iterate over to
find non-locked one.
3) I have job result
So, instead of
jobs/
 job1
  .lock
  .result
I have
jobs/
 job1 (holding data)
locks/
 job1
queue/
 job1 (holding unlocked jobs)
result/
 job1

This is  actually 1 more node per job (queue) to handle (2)

But, my main Q is about watches leakes. Even in your scheme, adding a watch
on jobs/job1/.result may produce a leak if job is cancelled before it was
processed (so that .result never filled).



-- 
Best regards,
 Vitalii Tymchyshyn

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message