Mailing-List: contact bookkeeper-user-help@zookeeper.apache.org; run by ezmlm
Precedence: bulk
Reply-To: bookkeeper-user@zookeeper.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
Date: Mon, 17 Sep 2012 11:07:16 +0200
From: Ivan Kelly <ivank@apache.org>
To: bookkeeper-user@zookeeper.apache.org
Subject: Re: Should I use Hedwig or BookKeeper?
Message-ID: <20120917090714.GK9668@moloch.vm.bytemark.co.uk>
References: 
 <CACcNepQzGnK+GJxz9nExOaO9xomMb6QB8izC1seh7sO+zAY0MA@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: 
 <CACcNepQzGnK+GJxz9nExOaO9xomMb6QB8izC1seh7sO+zAY0MA@mail.gmail.com>
User-Agent: Mutt/1.5.20 (2009-06-14)

Hi ChenWei,

HDFS namenode only reads journals which have been finalized, which is
the HDFS way to describe that they have been closed and no more
entries can be added. There's no mechanism at the moment for getting a
live stream of another namenode's updates. For this reason, we haven't
implemented one for BookKeeper journals, because it would be never
used.

Hedwig could have been used to solve the problem you described, but
there's no way to fit this into the current namenode journalling
architecture.

Hedwig may be a better option in your case, as from it will allow you
to keep hot standbys.

-Ivan

On Sun, Sep 16, 2012 at 12:01:15AM +0800, ChenWei wrote:
> Hello,
>   I need a reliable system that can store and distribute logs among several
> servers. It means, one server writes logs and others can get notice at once
> and read this log. Any one of these servers dies, logs can be recovered.
>   I was planing to use BookKeeper because my demand is very similar with
> that of HDFS namenode, but I found that there is no notice mechanism in it
> and when a server is writing logs, others can not read them safely. But
> Hedwig seems to be used to solve this problem. Do I comprehend them
> correctly?
>   But why HDFS namenode inspired BookKeeper instead of Hedwig?