Mailing-List: contact zookeeper-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: zookeeper-user@hadoop.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; s=serpent; d=yahoo-inc.com; c=nofws; q=dns;
	h=received:user-agent:date:subject:from:to:message-id:
	thread-topic:thread-index:in-reply-to:mime-version:content-type:
	content-transfer-encoding:return-path:x-originalarrivaltime;
	b=jxomSLnHKfFcs/fbjxg2XdIiE5YUMRPCTQW4qye9zOeU75tNGjMf3M0sJZYABCh0
User-Agent: Microsoft-Entourage/12.20.0.090605
Date: Sun, 24 Jan 2010 19:08:59 -0800
Subject: Re: Using zookeeper to assign a bunch of long-running tasks to nodes
 (without unhandled tasks and double-handled tasks)
From: Mahadev Konar <mahadev@yahoo-inc.com>
To: <zookeeper-user@hadoop.apache.org>
Message-ID: <C7824B4B.2D695%mahadev@yahoo-inc.com>
Thread-Topic: Using zookeeper to assign a bunch of long-running tasks to nodes
  (without unhandled tasks and double-handled tasks)
Thread-Index: Acqda7zqKhKWSGOngUG1wB9lwwgR2w==
In-Reply-To: <34fd060d1001230058ybeca0b1n79bdcd97e63be76@mail.gmail.com>
Mime-version: 1.0
Content-type: text/plain;
	charset="US-ASCII"
Content-transfer-encoding: 7bit

Hi  Zheng,
> Let's say I have 100 long-running tasks and 20 nodes.
> I want each of them to take up to 10 tasks. Each of the task should be
> taken by one and only one node.
This is exactly one of our users is using ZooKeeper for. You might want to
make it more general  saying that a directory /tasks/ will have the list of
tasks that need to be processed - (in your case 0-99). Basically storing the
list of tasks also in zookeeper. The clients can then read of this list and
try creating ephemeral nodes for tasks in mytasks/ and assign themselves as
the owner of those tasks.
You also should factor in the task dying or the machine not able to start
that task. In that case the machine should just remove the ephemeral node
that it created and should let the other machines take up that task.

Here is one minor thing that might be useful. One of the zookeeper users who
was doing exactly the same thing had the number of failures of booting up a
task stored as data in /tasks/ znode for that task. This way all the
machines can update this count and alert (to the admin) if a task cannot be
started or worked upon by a given count of machines.

Hope this helps.

Thanks
Mahadev


On 1/23/10 12:58 AM, "Zheng Shao" <zshao9@gmail.com> wrote:

> Let's say I have 100 long-running tasks and 20 nodes.
> I want each of them to take up to 10 tasks. Each of the task should be
> taken by one and only one node.
> 
> Will the following solution solve the problem?
> 
> Create a directory "/mytasks" in zookeeper.
> Normally there will be 100 EPHEMERAL children in /mytasks directory,
> named from "0" to "99".
> The data of each will be the name of the node and the process id in
> the node. This data is optional but allow us to do lookup from task to
> node and process id.
> 
> 
> Each node will start 10 processes.
>   Each process will list the directory "/mytasks" with a watcher
>   If trigger by the watcher, we relist the directory.
>   If we found some missing files in the range of "0" to "99", we
> create an EPHEMERAL node with no-overwrite option
>     if the creation is successful, then we disable the watcher and
> start processing the corresponding task (if something goes wrong, just
> kill itself and the node will be gone)
>     if not, we go back to wait for watcher.
> 
> Will this work?
> 
>