Mailing-List: contact zookeeper-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: zookeeper-user@hadoop.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; s=serpent; d=yahoo-inc.com; c=nofws; q=dns;
	h=received:from:to:cc:date:subject:thread-topic:
	thread-index:message-id:references:in-reply-to:accept-language:
	content-language:x-ms-has-attach:x-ms-tnef-correlator:acceptlanguage:
	content-type:content-transfer-encoding:mime-version;
	b=c+vIaXuWfeGre9/GqNSendCs1PZ8TM3drIm1MlLOa+mwkBoOkYZoT9BwLcCEzfQk
From: Benjamin Reed <breed@yahoo-inc.com>
To: "zookeeper-user@hadoop.apache.org" <zookeeper-user@hadoop.apache.org>
CC: "nuthanashkore@yahoo.com" <nuthanashkore@yahoo.com>
Date: Tue, 6 Jan 2009 10:44:08 -0800
Subject: RE: group messaging, empheral nodes on zookeeper
Thread-Topic: group messaging, empheral nodes on zookeeper
Thread-Index: AclwKDrK2QxRSgYwRz2juYtxBJ5HzQABodKy
Message-ID: 
 <6990D2A1CAF07E40A7CFE68A5FAAA15317F2D4AD7E@SP1-EX07VS02.ds.corp.yahoo.com>
References: <797838.36409.qm@web111011.mail.gq1.yahoo.com>
	 <30c6373b0901060928l4da83bf4s8590467a2409d972@mail.gmail.com>
	 <732AD923-EB4F-4FB9-A08D-566E6085D150@yahoo-inc.com>,<30c6373b0901060955k4c335511x41da72c52a3c09@mail.gmail.com>
In-Reply-To: <30c6373b0901060955k4c335511x41da72c52a3c09@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
acceptlanguage: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

Your first case only works if it is okay to miss messages as long as you se=
e the latest.

For your second case (using a directory with a child node for each message)=
 I agree with flavio that you do not want to use ephemeral nodes. you can i=
mplement a ttl by having a daemon process that periodically harvests old me=
ssages.

ben

________________________________________
From: burtonator@gmail.com [burtonator@gmail.com] On Behalf Of Kevin Burton=
 [burton@spinn3r.com]
Sent: Tuesday, January 06, 2009 9:55 AM
To: zookeeper-user@hadoop.apache.org
Cc: nuthanashkore@yahoo.com
Subject: Re: group messaging, empheral nodes on zookeeper

>
> In case 1), if you are proposing to overwrite the content of the znode,
> then you would need first to make sure that all receivers have already
> received the previous message. This doesn't seem a good solution to me
> because a client that wants to broadcast a message would have to wait unt=
il
> all others flag that they have received the previous message. Appending n=
ew
> messages to the current content of the znode doesn't seem a great solutio=
n
> either because clients would have to make sure that they are overwriting =
the
> correct version.
>

It's not good for all situations... we have a daemon state file we're using
and it works well for this use case.

Also the throughput here is very low.


>
> Solution 2) sounds better to me because it is wait-free: no client willin=
g
> to broadcast a message contends with other clients. Now, as you say, you
> have to use the sequence flag to make sure that all messages are delivere=
d
> in the same order (if you want total order). You may still want to have a
> mechanism for clients to flag which messages they have already delivered =
so
> that you remove messages that everyone has already delivered. If your
> application generates a large number of messages and you don't remove the=
m,
> you might end up with an unnecessarily large number of znodes. Finally, I=
'm
> not sure why you want to use ephemeral znodes for messages. I think that
> ephemeral znodes may cause you trouble in this case. Suppose that a clien=
t
> broadcasts a message by creating an ephemeral znode and crashes before al=
l
> other clients deliver the message. You may end up with two clients
> delivering different sequences of messages.
>

But you would need a cleanup mechanism if they weren't ephemeral...

Of course if ephemeral nodes weren't necessarily session based and had a TT=
L
value that would solve this problem.

Kevin

--
Founder/CEO Spinn3r.com
Location: San Francisco, CA
AIM/YIM: sfburtonator
Skype: burtonator
Work: http://spinn3r.com