hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suresh Srinivas (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-1975) HA: Support for sharing the namenode state from active to standby.
Date Wed, 24 Aug 2011 23:11:29 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090607#comment-13090607

Suresh Srinivas commented on HDFS-1975:

I and Jitendra considered Transaction ID before looking at GS. Transaction ID does not work
because there are three parties involved - client, datanode and namenode.

Take this example:
# DN1 sends heartbeat to primary NN at txid T and learns about T.
# A client meanwhile creates a file at T+1 and allocates block at T+2.
# DN1 now is unable to send heartbeat or communicate with primary NN. Hence it is stuck at
transaction T.
# Client complets writing a block to DN1. DN1 reports this to backup node as block received
with T. At this point in time, if SNN has reached T and has not processed T+1 or T+2, it tries
to handle BR(T), because it can. However, it fails to process it without the knowledge of
the file.

We could get around this, if client also is tracking transactions and sends it to the datanode,
adding unnecessary complexity and changes.

> HA: Support for sharing the namenode state from active to standby.
> ------------------------------------------------------------------
>                 Key: HDFS-1975
>                 URL: https://issues.apache.org/jira/browse/HDFS-1975
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>            Reporter: Suresh Srinivas
>            Assignee: Jitendra Nath Pandey
>         Attachments: hdfs-1975.txt, hdfs-1975.txt
> To enable hot standby namenode, the standby node must have current information for -
namenode state (image + edits) and block location information. This jira addresses keeping
the namenode state current in the standby node. To do this, the proposed solution in this
jira is to use a shared storage to store the namenode state. 
> Note one could also build an alternative solution by augmenting the backup node. A seperate
jira could explore this.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message