hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dhruba borthakur (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5189) Integration with BookKeeper logging system
Date Tue, 10 Feb 2009 00:09:59 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12672070#action_12672070

dhruba borthakur commented on HADOOP-5189:

Hi Luca, Thanks for your comments. It would be nice if we can enhance the current Namenode
code to support invoking logSync() from multiple threads. This will help namenode performance
even when it is writing edits-log to multiple files (and no bookKeeper is involed). If we
have a patch for HADOOP-5188 and we apply that first to the Hadoop trunk, then the bookKeeper
patch will fit  very elegantly. 

If the above is done as a pre-requisite for your bookKeeper patch, then we would be able to
support file based edits log as well as bookKeeper based transaction log as the same time.
This means that a namenode can log each transactions to a local file as well as bookKeeper.
This might help in transitioning current hdfs namenode to using bookKeeper.

> it's more complex and not sure if it's worth the effort, in principle because you'd have
to maintain two different logging system in parallel

I agree.

> Integration with BookKeeper logging system
> ------------------------------------------
>                 Key: HADOOP-5189
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5189
>             Project: Hadoop Core
>          Issue Type: New Feature
>    Affects Versions: 0.19.0
>            Reporter: Luca Telloli
>         Attachments: create.png, HADOOP-5189.patch
> BookKeeper is a system to reliably log streams of records (https://issues.apache.org/jira/browse/ZOOKEEPER-276).
The NameNode is a natural target for such a system for being the metadata repository of the
entire file system for HDFS. 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message