hadoop-zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kevin Burton" <bur...@spinn3r.com>
Subject Re: group messaging, empheral nodes on zookeeper
Date Tue, 06 Jan 2009 17:55:31 GMT
>
> In case 1), if you are proposing to overwrite the content of the znode,
> then you would need first to make sure that all receivers have already
> received the previous message. This doesn't seem a good solution to me
> because a client that wants to broadcast a message would have to wait until
> all others flag that they have received the previous message. Appending new
> messages to the current content of the znode doesn't seem a great solution
> either because clients would have to make sure that they are overwriting the
> correct version.
>

It's not good for all situations... we have a daemon state file we're using
and it works well for this use case.

Also the throughput here is very low.


>
> Solution 2) sounds better to me because it is wait-free: no client willing
> to broadcast a message contends with other clients. Now, as you say, you
> have to use the sequence flag to make sure that all messages are delivered
> in the same order (if you want total order). You may still want to have a
> mechanism for clients to flag which messages they have already delivered so
> that you remove messages that everyone has already delivered. If your
> application generates a large number of messages and you don't remove them,
> you might end up with an unnecessarily large number of znodes. Finally, I'm
> not sure why you want to use ephemeral znodes for messages. I think that
> ephemeral znodes may cause you trouble in this case. Suppose that a client
> broadcasts a message by creating an ephemeral znode and crashes before all
> other clients deliver the message. You may end up with two clients
> delivering different sequences of messages.
>

But you would need a cleanup mechanism if they weren't ephemeral...

Of course if ephemeral nodes weren't necessarily session based and had a TTL
value that would solve this problem.

Kevin

-- 
Founder/CEO Spinn3r.com
Location: San Francisco, CA
AIM/YIM: sfburtonator
Skype: burtonator
Work: http://spinn3r.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message