hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dhruba borthakur (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-839) The NameNode should forward block reports to BackupNode
Date Fri, 15 Jan 2010 22:44:54 GMT

    [ https://issues.apache.org/jira/browse/HDFS-839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12800970#action_12800970

dhruba borthakur commented on HDFS-839:

> what's the rationale for doing so?

I am thinking that the namenode has to spend considerable horsepower to keep forwarding block
reports and blocksReceived to BackupNode. I would like to have a HA solution without impacting
the performance of the NN. Thus, the proposal to make datanodes send blocks reports and blocks
received to NN as well as BN.

> the backup node can just re-replicate any blocks it thinks are under-replicated when
it becomes the primary.


However, when we fail over from NN to BN,  there could be blocks in the BN that we should
quickly find if there are any blocks in the BN that do not have any valid replicas and then
query all datanodes to send a block report for those blocks.

> The NameNode should forward block reports to BackupNode
> -------------------------------------------------------
>                 Key: HDFS-839
>                 URL: https://issues.apache.org/jira/browse/HDFS-839
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: name-node
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
> The BackupNode (via HADOOP-4539) receives a stream of transactions from NameNode. However,
the BackupNode does not have block locations of blocks. It would be nice if the NameNode can
forward all block reports (that it receives from DataNodes) to the BackupNode.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message