hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suresh Srinivas (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3077) Quorum-based protocol for reading and writing edit logs
Date Tue, 03 Apr 2012 20:36:25 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13245702#comment-13245702

Suresh Srinivas commented on HDFS-3077:

bq. I prefer JournalNode because every other daemon we have is a *Node. If you're running
it inside another process, I think we would just call it a "JournalService" – or an "embedded
JournalNode". I think of a daemon as a standalone process.
I think that is fine. I have initial JournalService that is implemented as a part of 3092.
We will consolidate from your patch, this part of the code in HDFS-3178.

bq. OK. This part I have done in the patch attached here and works pretty well, so far. If
you want, I'm happy to separate out the quorum completion code to commit it ASAP so we can
share code here.
This sounds good.

bq. I think the "standalone" nature of the approach outweighs what benefit we might get by
reusing ZK.
We can look into this in more detail. However, we will add a method called fence() in the
JournalProtocol, with epoch number.

I will get back to you on the last comment.

> Quorum-based protocol for reading and writing edit logs
> -------------------------------------------------------
>                 Key: HDFS-3077
>                 URL: https://issues.apache.org/jira/browse/HDFS-3077
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: ha, name-node
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>         Attachments: hdfs-3077-partial.txt, qjournal-design.pdf
> Currently, one of the weak points of the HA design is that it relies on shared storage
such as an NFS filer for the shared edit log. One alternative that has been proposed is to
depend on BookKeeper, a ZooKeeper subproject which provides a highly available replicated
edit log on commodity hardware. This JIRA is to implement another alternative, based on a
quorum commit protocol, integrated more tightly in HDFS and with the requirements driven only
by HDFS's needs rather than more generic use cases. More details to follow.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message