hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Collins (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-839) The NameNode should forward block reports to BackupNode
Date Fri, 15 Jan 2010 16:48:54 GMT

    [ https://issues.apache.org/jira/browse/HDFS-839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12800750#action_12800750

Eli Collins commented on HDFS-839:

bq. I would rather make the datanode send block reports and block received messages directly
to the master as well as the BackupNode. do you agree?

Once nice thing about having the namenode sync to the backup node is that there should be
less divergence in state (eg the block  maps) so you have an easier coherency problem between
the primary and the backup. Can't think of why divergence would necessarily be hard to deal
with though, eg the backup node can just re-replicate any blocks it thinks are under-replicated
when it becomes the primary.

Also wondering aloud if it would make implementing snapshots easier if the namenode gets to
update the backup node state, eg could do so atomically. 

Having the datanodes send block reports/received directly to the backup node seems like it
could work as well, what's the rationale for doing so?

> The NameNode should forward block reports to BackupNode
> -------------------------------------------------------
>                 Key: HDFS-839
>                 URL: https://issues.apache.org/jira/browse/HDFS-839
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: name-node
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
> The BackupNode (via HADOOP-4539) receives a stream of transactions from NameNode. However,
the BackupNode does not have block locations of blocks. It would be nice if the NameNode can
forward all block reports (that it receives from DataNodes) to the BackupNode.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message