hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-1975) HA: Support for sharing the namenode state from active to standby.
Date Thu, 24 Nov 2011 06:56:40 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13156568#comment-13156568
] 

Todd Lipcon commented on HDFS-1975:
-----------------------------------

I'm trying to write a test that combines this with HDFS-1971, HDFS-2591, and HDFS-1108, and
ran into an issue with the design:

With HDFS-1108, we persist blocks, but the order of the edits in the edit log is:
{code}
2: SET_GENERATION_STAMP 1001
3: ADD /testStandbyIsHot with no blocks
4: SET_GENERATION_STAMP 1002
5: ADD /testStandbyIsHot with the block id blk_486254483591558448_1002 length=0
6: CLOSE /testStandbyIsHot with the block id blk_486254483591558448_1002 length=1000
{code}

the issue is that, when the StandbyNode is receiving the edits, it gets SET_GENERATION_STAMP=1002
_before_ it gets the mapping of that block into the file. So, it processes the blockReceived,
sees it as an invalid block not belonging to any file, and ignores it.

Will think about a solution....
                
> HA: Support for sharing the namenode state from active to standby.
> ------------------------------------------------------------------
>
>                 Key: HDFS-1975
>                 URL: https://issues.apache.org/jira/browse/HDFS-1975
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>            Reporter: Suresh Srinivas
>            Assignee: Jitendra Nath Pandey
>         Attachments: HDFS-1975-HA.2.patch, HDFS-1975-HA.patch, HDFS-1975-HDFS-1623.patch,
HDFS-1975-HDFS-1623.patch, hdfs-1975.txt, hdfs-1975.txt
>
>
> To enable hot standby namenode, the standby node must have current information for -
namenode state (image + edits) and block location information. This jira addresses keeping
the namenode state current in the standby node. To do this, the proposed solution in this
jira is to use a shared storage to store the namenode state. 
> Note one could also build an alternative solution by augmenting the backup node. A seperate
jira could explore this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message