hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dhruba borthakur (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-1431) Balancer should work with the logic of BlockPlacementPolicy
Date Sat, 02 Oct 2010 06:53:34 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12917165#action_12917165

dhruba borthakur commented on HDFS-1431:

> This method will have to act as if it works outside of the NameNode which is different
form all other methods.

It can make RPC calls to the NN to get the information needed to make this decision. do you
think it will add significant load on the NN? If so, it can make a bulk-RPC to the NN to fetch
all the required information for all the relevant blocks in one-shot.

> Balancer should work with the logic of BlockPlacementPolicy
> -----------------------------------------------------------
>                 Key: HDFS-1431
>                 URL: https://issues.apache.org/jira/browse/HDFS-1431
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: balancer
>    Affects Versions: 0.22.0
>            Reporter: Scott Chen
>            Assignee: Scott Chen
>             Fix For: 0.22.0
>         Attachments: HDFS-1431.txt
> Currently Balancer does not obtain information from BlockPlacementPolicy so it can transfer
the blocks without checking with BlockPlacementPolicy.
> This causes the policy break after balancing the cluster.
> There are some new policies proposed in HDFS-1094 and MAPREDUCE-1831 in which the block
placement follows some pattern.
> The pattern can be broken by Balancer.
> I propose that we add the following method in BlockPlacementPolicy:
> {code}
>   abstract public boolean canBeMoved(String fileName, Block block,
>     DatanodeInfo source, DatanodeInfo destination);
> {code}
> And make Balancer use it in
> {code}
>   private boolean isGoodBlockCandidate(Source source,
>       BalancerDatanode target, BalancerBlock block)
> {code}
> What do you think?

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message