hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brandon Li (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3425) Enable standby namenode to tail edits from journal service
Date Tue, 19 Jun 2012 19:40:43 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13397014#comment-13397014
] 

Brandon Li commented on HDFS-3425:
----------------------------------

Todd, Thanks for explaining your approach in details. Using the same interface reading edit
logs minimizes the code change on the namenode side. I planed to stream the edit logs in HDFS-3033.
It's good that you already have it implemented in your branch. 

I suspect you still need the "if(useJournalService())" alike calls when HA is considered later
on. My understanding of the namenode start procedure with JournalService is like the following
roughly:
1. namenode inits (in standby mode)
2. load fsimage
3. tail finalized segments
4. transit to active mode
5. collect last txnId from quorum number of JD (to find the txnId to roll edit)
6. roll edit, at east quorum number of JDs in sync
7. load the last (1 or 0) finalized edit segment

The namenode has to be aware of the HA approaches, shared edits or using journal service.

                
> Enable standby namenode to tail edits from journal service
> ----------------------------------------------------------
>
>                 Key: HDFS-3425
>                 URL: https://issues.apache.org/jira/browse/HDFS-3425
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ha, name-node
>            Reporter: Brandon Li
>            Assignee: Brandon Li
>         Attachments: HDFS-3425.HDFS-3092.patch, HDFS-3425.HDFS-3092.patch.2
>
>
> In an HA cluster with the journal service configured, namenode should get edit logs from
journal service instead of from local or shared storage directories.  
> As one of the first steps towards this goal, this JIRA is to enable standby namenode
to tail edits from journal services. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message