hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tom White (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3799) Design a pluggable interface to place replicas of blocks in HDFS
Date Fri, 19 Jun 2009 15:04:07 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12721805#action_12721805

Tom White commented on HADOOP-3799:

Hi Dhruba,

A couple more comments:

> BlockPlacementInterface#chooseTarget(). Make excludedNodes a List<DatanodeDescriptor>.
Implementations may choose to turn it into a map if they need to, but for the interface, it
should just be a list, shouldn't it?

I think you missed this change.

I'm not convinced that ReplicationPolicyChooser is needed. Couldn't we add a static method
(e.g. getInstance()) to BlockPlacementPolicy to construct a BlockPlacementPolicy from the
dfs.block.replicator.classname property? We can add an overloaded chooseTarget() method to
BlockPlacementPolicy which doesn't take a chosenNodes argument (BTW this is misspelt as "choosenNodes"
in BlockPlacementPolicy).

> Design a pluggable interface to place replicas of blocks in HDFS
> ----------------------------------------------------------------
>                 Key: HADOOP-3799
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3799
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: dfs
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>         Attachments: BlockPlacementPluggable.txt, BlockPlacementPluggable2.txt
> The current HDFS code typically places one replica on local rack, the second replica
on remote random rack and the third replica on a random node of that remote rack. This algorithm
is baked in the NameNode's code. It would be nice to make the block placement algorithm a
pluggable interface. This will allow experimentation of different placement algorithms based
on workloads, availability guarantees and failure models.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message