hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Luca Telloli (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5189) Integration with BookKeeper logging system
Date Mon, 09 Feb 2009 10:29:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12671794#action_12671794

Luca Telloli commented on HADOOP-5189:

Jeff, I think that for being a logging abstraction, BookKeeper does not implement it but provides
a way to achieve it. That is, if the NameNode dies, another process could reconstruct the
fsimage starting from a set of ledgers. Actually, any number of independent processes could
do it, as soon as they read from the same set of ledgers. This process could definitely be
the SecondaryNameNode.
I guess that such a process could be implemented to allow warm standby, as proposed in the
description. First thought: ZooKeeper could be used to store such an information (the set
of ledger IDs that belong to a single fsimage) and the secondary NameNodes could periodically
poll for changes (or maybe using some notification mechanism). 

> Integration with BookKeeper logging system
> ------------------------------------------
>                 Key: HADOOP-5189
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5189
>             Project: Hadoop Core
>          Issue Type: New Feature
>    Affects Versions: 0.19.0
>            Reporter: Luca Telloli
>         Attachments: create.png, HADOOP-5189.patch
> BookKeeper is a system to reliably log streams of records (https://issues.apache.org/jira/browse/ZOOKEEPER-276).
The NameNode is a natural target for such a system for being the metadata repository of the
entire file system for HDFS. 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message