zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bruno Freudensprung <bfreudenspr...@expertsystem.com>
Subject RE: Observers: a link to message bus?
Date Wed, 24 Jan 2018 13:59:15 GMT
Hi Andor,

Thanks a lot for your answer!
I have another question if you don't mind. Once again I am just curious about Zookeeper and
I don't want to waste your time, so please simply ignore this message if it is the case. 
Hacking the ObserverZooKeeperServer (hooking custom code when INFORM messages are commited)
would enable a solution based on changing the value of the same zk node, over and over. In
that case the number of zk nodes will remain small, even if "change throughput" is high. However
a solution based on observer+client will have to be based on adding new (ephemeral) nodes
to the same zk node (otherwise the client might miss some updates). So I'll have to find a
way to "timeout" and remove those nodes at some point. Does that sound correct?


-----Message d'origine-----
De : Andor Molnar [mailto:andor@cloudera.com] 
Envoyé : mercredi 24 janvier 2018 11:16
À : user@zookeeper.apache.org
Objet : Re: Observers: a link to message bus?

Hi Bruno,

Unfortunately I have no idea what plugin system does the author of this doc refer to, but
I can think of the following 2 solutions:

1) Fire up some observers in ZK cluster and connect to them via some standard client. Create
a zk node and create a watch for its children.
Producer can create nodes as children of this special node and connected clients (consumers)
can receive them in an ordered manner. This setup will not affect the overall performance
of the cluster, because the 'message bus' only uses observers to achieve its goal.

2) Write your custom Observer node by implementing ZK protocol that can receive and parse
INFORM messages. These messages are consumed by Observers too. Obviously this solution could
be enormously cumbersome, although Observers are the least complicated nodes of a ZK ensemble,
but I'm not aware of any existing 'plugin' solution which would achieve similar forwarder


On Wed, Jan 17, 2018 at 2:57 PM, Martin Gainty <mgainty@hotmail.com> wrote:

> ________________________________
> From: Bruno Freudensprung <bfreudensprung@expertsystem.com>
> Sent: Tuesday, January 16, 2018 7:54 AM
> To: user@zookeeper.apache.org
> Subject: Observers: a link to message bus?
> Hi,
> I have a question regarding this part of the Observers Guide (
> https://zookeeper.apache.org/doc/r3.4.11/zookeeperObservers.html):
> ZooKeeper Observers<https://zookeeper.apache.org/doc/r3.4.11/
> zookeeperObservers.html>
> zookeeper.apache.org
> Setting up a ZooKeeper ensemble that uses Observers is very simple, 
> and requires just two changes to your config files. Firstly, in the 
> config file of every node ...
> "As a link to a message bus : [...] Observers would give a natural 
> integration point for this work: a plug-in mechanism could be used to 
> attach the stream of proposals an Observer sees to a publish-subscribe 
> system"
> I've not been able to find such a plug-in mechanism in the source code 
> of ZooKeeper. For the moment the only integration point I can think of 
> is the commitRequest method of the  
> org.apache.zookeeper.server.quorum.ObserverZooKeeperServer
> class. Am I looking in the right direction?
> To be honest: this is for the moment only pure curiosity since Apache 
> Kafka seems to be a more "natural" and out-of-the-box option for an 
> ordered message bus. However since I might probably need ZooKeeper for 
> its distributed lock and leader election features, I was wondering if 
> I could use it as an ordered message bus as well.
> MG>if ActiveMQ does indeed support ordered messaging for multiple 
> MG>active
> consumers thru Message Groups as stated:
> MG>Message Groups which is the way to load balance multiple active
> consumers listening to the same queue while preserving message order.
> MG>i have asked why does Zookeeper not support 
> MG>MessageOrderingTechnology
> such as ActiveMQ  as a plugin?
> MG>have you discovered "plugin integration" within the Zookeeper
> Architecture?
> Best regards,
> Bruno.
> MG>Thanks Bruno
View raw message