Return-Path: 
 <user-return-2746-apmail-zookeeper-user-archive=zookeeper.apache.org@zookeeper.apache.org>
Delivered-To: apmail-zookeeper-user-archive@www.apache.org
Received: (qmail 69629 invoked from network); 13 Feb 2011 16:11:01 -0000
Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3)
  by minotaur.apache.org with SMTP; 13 Feb 2011 16:11:01 -0000
Received: (qmail 73437 invoked by uid 500); 13 Feb 2011 16:11:00 -0000
Delivered-To: apmail-zookeeper-user-archive@zookeeper.apache.org
Received: (qmail 73284 invoked by uid 500); 13 Feb 2011 16:10:58 -0000
Mailing-List: contact user-help@zookeeper.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:user-help@zookeeper.apache.org>
List-Unsubscribe: <mailto:user-unsubscribe@zookeeper.apache.org>
List-Post: <mailto:user@zookeeper.apache.org>
List-Id: <user.zookeeper.apache.org>
Reply-To: user@zookeeper.apache.org
Delivered-To: mailing list user@zookeeper.apache.org
Received: (qmail 73276 invoked by uid 99); 13 Feb 2011 16:10:57 -0000
Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
    by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 13 Feb 2011 16:10:57 +0000
X-ASF-Spam-Status: No, hits=-0.0 required=5.0
	tests=SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (athena.apache.org: local policy)
Received: from [88.198.2.104] (HELO koch.ro) (88.198.2.104)
    by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 13 Feb 2011 16:10:51 +0000
Received: from 240-24.79-83.cust.bluewin.ch ([83.79.24.240]
 helo=jona.localnet)
	by koch.ro with esmtpsa (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.69)
	(envelope-from <thomas@koch.ro>)
	id 1PoeWr-0000FS-SX; Sun, 13 Feb 2011 17:10:30 +0100
From: Thomas Koch <thomas@koch.ro>
Reply-To: thomas@koch.ro
To: user@zookeeper.apache.org
Subject: Re: ExecutorService over a zookeeper
Date: Sun, 13 Feb 2011 17:10:20 +0100
User-Agent: KMail/1.13.5 (Linux/2.6.32-4-amd64; KDE/4.4.5; x86_64; ; )
Cc: =?utf-8?b?0JLRltGC0LDQu9GW0Lkg0KLQuNC80YfQuNGI0LjQvQ==?=
 <tivv00@gmail.com>
References: <AANLkTinvXRKWKEg6E6UcG=Q7pyFt10Gmk63FSykEr2-1@mail.gmail.com>
In-Reply-To: <AANLkTinvXRKWKEg6E6UcG=Q7pyFt10Gmk63FSykEr2-1@mail.gmail.com>
MIME-Version: 1.0
Content-Type: Text/Plain;
  charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Message-Id: <201102131710.21091.thomas@koch.ro>

=D0=92=D1=96=D1=82=D0=B0=D0=BB=D1=96=D0=B9 =D0=A2=D0=B8=D0=BC=D1=87=D0=B8=
=D1=88=D0=B8=D0=BD:
> Hello.
>=20
> I am trying to implement distributed executor service over a zookeeper and
> encountered a problem I am not sure how to deal with.
> <SNIP />
Hi,

I've the impression, you're doing it too complicate. Why not have a znode=20
jobs/ and put all your jobs as persistent znodes in there.
So your job_queue could look like:

jobs/
  job1
  job2
  job3

A consumer can then lock a job by putting an ephemeral node in one of the j=
ob=20
znodes:

jobs/
  job1
  job2/
    .lock
  job3

Other consumers see that somebody is already working on job2. If the consum=
er=20
should die, the ephemeral lock will vanish and another consumer could take=
=20
over.
You should not publish results in ZK but use some other communication metho=
d.=20
ZK is really not meant as a messaging solution.
Maybe you'd be better served with a standard message queue (e.g. Apache=20
activeMQ, rabbitMQ) and use ZK only for coordination? If you have really hi=
gh=20
message load, you could have a look at Hedwig (google://zookeeper hedwig).

Best regards,

Thomas Koch, http://www.koch.ro