Mailing-List: contact bookkeeper-user-help@zookeeper.apache.org; run by ezmlm
Precedence: bulk
Reply-To: bookkeeper-user@zookeeper.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
Date: Thu, 3 May 2012 19:25:34 +0200
From: Ivan Kelly <ivank@apache.org>
To: bookkeeper-user@zookeeper.apache.org
Subject: Re: topic naming, grouping of messages and topic redistribution in
 Hedwig
Message-ID: <20120503172533.GG10622@moloch.vm.bytemark.co.uk>
References: <4FA26705.5050906@apache.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <4FA26705.5050906@apache.org>
User-Agent: Mutt/1.5.20 (2009-06-14)

> -  Are topic names unique across Hedwig instances (spread on
> different datacenters for example)? Is there a naming scheme?
No, a topic name is global to the system. If you publish to TopicA on
region1, a subscriber on region2 will get the message.


> - How can one ensure that a group of messages can be handled
> atomically by Hedwig (in particular 1 message persisted => all
> messages from the group persisted)? One way could be to create a
> single message containing all messages from the group, and send that
> to Hedwig, but maybe there is a better way?
Batching into one message is the only way to do this. 

> - when a hub fails, its topics are redistributed to other hubs, but
> would it be possible to control the redistribution? This could be
> useful in order to limit the number of hubs a client would have to
> connect to (in the case all topics are reallocated to distinct
> hubs). It seems that the current logic is in ZkTopicManager and
> redistribution of topic is based on load information gathered from
> the available hubs. But could this be made customizable?
There's actually work planned to make the TopicManager interfaces more
pluggable so that other backends could be plugged in. This work could
be folded into that maybe.

-Ivan