zookeeper-bookkeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stone <stones....@gmail.com>
Subject Re: Is there any way to set a string name for a ledger and how to do checkpoint ?
Date Wed, 30 May 2012 12:24:25 GMT
Thanks for your advice Ivan. It's very helpful.

Actually, I am experimenting write a In Memory Batch Store for write-behind
to reduce the load of DB and found that BookKeeper seems exactly what I
need for crash recovery.  Any suggestions about the write-behind approach ?

Best Regards,

On Wed, May 30, 2012 at 7:13 PM, Ivan Kelly <ivank@apache.org> wrote:

> Hi Stone,
> Currently the bookkeeper API only provides the primatives, and
> developers need to add their own glue code on top of this.
> It may be worth your while to take a look at the WAL implementation we
> did for HDFS[1]. This has "named" ledgers, and allow supports the
> checkpointing.
> A "editlog" is a sequence of ledgers, which the journal manager
> maintains in zookeeper. When a checkpoint is made, the current ledger
> is closed, and a new one is created. HDFS maintains it's own
> transaction ids so that it can removed old ledgers which are no longer
> needed after checkpointing.
> We have been thinking about pulling out this abstract to make it
> easier for general use, but not much progress has been made on this
> yet.
> Regards
> Ivan
> [1]
> https://github.com/apache/hadoop-common/tree/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal

View raw message