bookkeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Uma Maheswara Rao G <hadoop....@gmail.com>
Subject Re: Should I use Hedwig or BookKeeper?
Date Mon, 17 Sep 2012 16:32:19 GMT
Adding to Ivan's point.

Yes, currently StandBy namenode will tail the edits
periodically(2mins). So, Standby will have the state almost up to
date, but not 100%. At max it will have 2mins edits less. On switch,
these edits will be catches up. It will take very less time, can
neglect.
So, in NN case, there is no need of notifying to standby on entry
addition. Notifying for each entry may create another overhead
As Ivan said, for your case, Hedwig may be the better option.

Regards,
Uma
On Mon, Sep 17, 2012 at 2:37 PM, Ivan Kelly <ivank@apache.org> wrote:
> Hi ChenWei,
>
> HDFS namenode only reads journals which have been finalized, which is
> the HDFS way to describe that they have been closed and no more
> entries can be added. There's no mechanism at the moment for getting a
> live stream of another namenode's updates. For this reason, we haven't
> implemented one for BookKeeper journals, because it would be never
> used.
>
> Hedwig could have been used to solve the problem you described, but
> there's no way to fit this into the current namenode journalling
> architecture.
>
> Hedwig may be a better option in your case, as from it will allow you
> to keep hot standbys.
>
> -Ivan
>
> On Sun, Sep 16, 2012 at 12:01:15AM +0800, ChenWei wrote:
>> Hello,
>>   I need a reliable system that can store and distribute logs among several
>> servers. It means, one server writes logs and others can get notice at once
>> and read this log. Any one of these servers dies, logs can be recovered.
>>   I was planing to use BookKeeper because my demand is very similar with
>> that of HDFS namenode, but I found that there is no notice mechanism in it
>> and when a server is writing logs, others can not read them safely. But
>> Hedwig seems to be used to solve this problem. Do I comprehend them
>> correctly?
>>   But why HDFS namenode inspired BookKeeper instead of Hedwig?

Mime
View raw message