hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-1975) HA: Support for sharing the namenode state from active to standby.
Date Wed, 24 Aug 2011 01:09:30 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089929#comment-13089929

Todd Lipcon commented on HDFS-1975:

Hi Jitendra. If you plan to use the generation stamp to synchronize the DN's block info with
the edit stream, I think we need to add {{nextGenerationStamp}} calls in a few places. In
particular, in allocateBlock, we don't bump the generation stamp, so the second block of a
new file will have the same GS as the first block if no other actions happen.

Have you enumerated the various coordinations that we need to consider? The above deals with
allocateBlock (namespace op) vs blockReceived (dn op), but I wonder if there are other places
we need to synchronize the DN action after some NN action and aren't bumping the GS.

One thing I was considering was threading transaction IDs through the various operations -
for example one possibility is for the active NN to send the current txid to the DN in every
DatanodeCommand. Then, any reports from DN->NN include that txid, and the standby can block
until it's hit that txid. Using txid instead of generation stamp means we don't have to consider
each type of operation as a special case. Thoughts?

> HA: Support for sharing the namenode state from active to standby.
> ------------------------------------------------------------------
>                 Key: HDFS-1975
>                 URL: https://issues.apache.org/jira/browse/HDFS-1975
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>            Reporter: Suresh Srinivas
>            Assignee: Jitendra Nath Pandey
>         Attachments: hdfs-1975.txt, hdfs-1975.txt
> To enable hot standby namenode, the standby node must have current information for -
namenode state (image + edits) and block location information. This jira addresses keeping
the namenode state current in the standby node. To do this, the proposed solution in this
jira is to use a shared storage to store the namenode state. 
> Note one could also build an alternative solution by augmenting the backup node. A seperate
jira could explore this.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message