hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Luca Telloli (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5189) Integration with BookKeeper logging system
Date Mon, 09 Feb 2009 23:02:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12672045#action_12672045

Luca Telloli commented on HADOOP-5189:

Dhruba, I found the EditLogInput/OutputStream interfaces insufficient for what we wanted to
achieve with BookKeeper, and because of this I separately filed https://issues.apache.org/jira/browse/HADOOP-5188
In this BookKeeper patch I modify the LogSync() method to allow parallel asynchronous writing
from multiple threads, while in the current logging system there's a single thread writing
at any time. We couldn't achieve as good results as we have with just a single thread, thus
the need for 5188.

With respect to maintaining the file-based logging I'm not sure if you mean the ability to
switch from one way of logging to the other, or for having them in parallel. The second one
I think it's more complex and not sure if it's worth the effort, in principle because you'd
have to maintain two different logging system in parallel and ensure that they are both consistent.
If you mean the first one, I'm thinking about a fix. 

> Integration with BookKeeper logging system
> ------------------------------------------
>                 Key: HADOOP-5189
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5189
>             Project: Hadoop Core
>          Issue Type: New Feature
>    Affects Versions: 0.19.0
>            Reporter: Luca Telloli
>         Attachments: create.png, HADOOP-5189.patch
> BookKeeper is a system to reliably log streams of records (https://issues.apache.org/jira/browse/ZOOKEEPER-276).
The NameNode is a natural target for such a system for being the metadata repository of the
entire file system for HDFS. 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message