zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From amit jaiswal <amit_...@yahoo.com>
Subject Question on production readiness, deployment, data of BookKeeper / Hedwig
Date Wed, 06 Oct 2010 05:32:54 GMT

In Hedwig talk (http://vimeo.com/13282102), it was mentioned that the primary 
use case for Hedwig comes from the distributed key-value store PNUTS in Yahoo!, 
but also said that the work is new.

Could you please about the following:

Production readiness / Deployment
1. What is the production readiness of Hedwig / BookKeeper. Is it being used 
anywhere (like in PNUTS)?
2. Is Hedwig designed to use as a generic message bus or only for 
multi-datacenter operations?
3. Hedwig installation and deployment is done through a script hw.bash, but that 
is difficult to use especially in a production environment. Are there any other 
packages available that can simplify the deployment of hedwig.
4. How does BK/Hedwig handle zookeeper session expiry?

Data Deletion, Handling data loss, Quorum
1. Does BookKeeper support deletion of old log entries which have been consumed.
2. How does Hedwig handles the case when all subscribers have consumed all the 
messages. In the talk, it was said that a subscriber can come back after hours, 
days or weeks. Is there any data retention / expiration policy for the data that 
is published?
3. How does Hedwig handles data loss? There is a replication factor, and a write 
operation must be accepted by majority of the bookies, but how data conflicts 
are handled? Is there any possibility of data conflict at all? Is the 
replication only for recovery? When the hub is reading data from bookies, does 
it reads from all the bookies to satisfy quorum read?

What is the difference between PubSubServer, HedwigSubscriber, 
HedwigHubSubscriber. Is there any HelloWorld program that simply illustrates how 
to instantiate a hedwig client, and publish/consume messages. (HedwigBenchmark 
class is helpful, but was looking something like API documentation).


View raw message