hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suresh Srinivas (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3092) Enable journal protocol based editlog streaming for standby namenode
Date Mon, 02 Apr 2012 17:33:25 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13244361#comment-13244361
] 

Suresh Srinivas commented on HDFS-3092:
---------------------------------------

bq. Hi Suresh. I took a look at the design document, and I think it actually shares a lot
with what I'm doing in HDFS-3077. Hopefully we can share some portions of the code and design.
Sounds good. Will add more details on how the code will be organized, so we can better reuse
the code.

bq. fencing command ensure that prior NNs can no longer access the JD after it completes
Fence command will include a version number that we got from the JournalList ZK node. The
number that is higher wins at the JD. The fence command with lower version # is rejected.

For the scenario you described, NN2 after it rolls JD2 and JD3, updates the JournalList with
JD2 and JD3. JD1 will no longer be used.

bq. Here are some points I think need elaboration in the design doc:
We decided to keep the document light to ensure the details do not distract from core mechanism.
Will add more details in the next version, including some use cases.

                
> Enable journal protocol based editlog streaming for standby namenode
> --------------------------------------------------------------------
>
>                 Key: HDFS-3092
>                 URL: https://issues.apache.org/jira/browse/HDFS-3092
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: ha, name-node
>    Affects Versions: 0.24.0, 0.23.3
>            Reporter: Suresh Srinivas
>            Assignee: Suresh Srinivas
>         Attachments: MultipleSharedJournals.pdf
>
>
> Currently standby namenode relies on reading shared editlogs to stay current with the
active namenode, for namespace changes. BackupNode used streaming edits from active namenode
for doing the same. This jira is to explore using journal protocol based editlog streams for
the standby namenode. A daemon in standby will get the editlogs from the active and write
it to local edits. To begin with, the existing standby mechanism of reading from a file, will
continue to be used, instead of from shared edits, from the local edits.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message