hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo (Nicholas), SZE (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-385) Design a pluggable interface to place replicas of blocks in HDFS
Date Wed, 01 Aug 2012 10:25:03 GMT

    [ https://issues.apache.org/jira/browse/HDFS-385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13426495#comment-13426495

Tsz Wo (Nicholas), SZE commented on HDFS-385:

Thanks for the update.  Some comments:

- Add "srcPath" as a parameter of FSNamesystem.chooseDatanode(..) and then pass "path" in

- Add the following comments to FSNamesystem.computeReplicationWorkForBlock(..) as in the
original patch.
     // choose replication targets: NOT HOLDING THE GLOBAL LOCK
+    // It is costly to extract the filename for which chooseTargets is called,
+    // so for now we pass in the Inode itself.

- Please remove the tabs since we do not use tabs in Hadoop.  Indentation is two spaces.

Please also run all the unit tests (ant test-core) since Jenkins doesn't work for branch-1.
> Design a pluggable interface to place replicas of blocks in HDFS
> ----------------------------------------------------------------
>                 Key: HDFS-385
>                 URL: https://issues.apache.org/jira/browse/HDFS-385
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>             Fix For: 0.21.0
>         Attachments: BlockPlacementPluggable.txt, BlockPlacementPluggable2.txt, BlockPlacementPluggable3.txt,
BlockPlacementPluggable4.txt, BlockPlacementPluggable4.txt, BlockPlacementPluggable5.txt,
BlockPlacementPluggable6.txt, BlockPlacementPluggable7.txt, blockplacementpolicy-branch-1-win.patch,
blockplacementpolicy-branch-1.patch, blockplacementpolicy2-branch-1-win.patch, blockplacementpolicy2-branch-1.patch,
> The current HDFS code typically places one replica on local rack, the second replica
on remote random rack and the third replica on a random node of that remote rack. This algorithm
is baked in the NameNode's code. It would be nice to make the block placement algorithm a
pluggable interface. This will allow experimentation of different placement algorithms based
on workloads, availability guarantees and failure models.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message