zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benjamin Reed <br...@yahoo-inc.com>
Subject RE: group messaging, empheral nodes on zookeeper
Date Tue, 06 Jan 2009 18:44:08 GMT
Your first case only works if it is okay to miss messages as long as you see the latest.

For your second case (using a directory with a child node for each message) I agree with flavio
that you do not want to use ephemeral nodes. you can implement a ttl by having a daemon process
that periodically harvests old messages.


From: burtonator@gmail.com [burtonator@gmail.com] On Behalf Of Kevin Burton [burton@spinn3r.com]
Sent: Tuesday, January 06, 2009 9:55 AM
To: zookeeper-user@hadoop.apache.org
Cc: nuthanashkore@yahoo.com
Subject: Re: group messaging, empheral nodes on zookeeper

> In case 1), if you are proposing to overwrite the content of the znode,
> then you would need first to make sure that all receivers have already
> received the previous message. This doesn't seem a good solution to me
> because a client that wants to broadcast a message would have to wait until
> all others flag that they have received the previous message. Appending new
> messages to the current content of the znode doesn't seem a great solution
> either because clients would have to make sure that they are overwriting the
> correct version.

It's not good for all situations... we have a daemon state file we're using
and it works well for this use case.

Also the throughput here is very low.

> Solution 2) sounds better to me because it is wait-free: no client willing
> to broadcast a message contends with other clients. Now, as you say, you
> have to use the sequence flag to make sure that all messages are delivered
> in the same order (if you want total order). You may still want to have a
> mechanism for clients to flag which messages they have already delivered so
> that you remove messages that everyone has already delivered. If your
> application generates a large number of messages and you don't remove them,
> you might end up with an unnecessarily large number of znodes. Finally, I'm
> not sure why you want to use ephemeral znodes for messages. I think that
> ephemeral znodes may cause you trouble in this case. Suppose that a client
> broadcasts a message by creating an ephemeral znode and crashes before all
> other clients deliver the message. You may end up with two clients
> delivering different sequences of messages.

But you would need a cleanup mechanism if they weren't ephemeral...

Of course if ephemeral nodes weren't necessarily session based and had a TTL
value that would solve this problem.


Founder/CEO Spinn3r.com
Location: San Francisco, CA
AIM/YIM: sfburtonator
Skype: burtonator
Work: http://spinn3r.com

View raw message