hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Scott Chen (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-1431) Balancer should work with the logic of BlockPlacementPolicy
Date Thu, 30 Sep 2010 23:51:36 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12916735#action_12916735

Scott Chen commented on HDFS-1431:

Dmytro: That's a good point.
How about this? We can create an interface like
interface BlockMovePolicy {
  boolean canBeMoved(...);
And make BlockPlacementPolicy implement the interface.
In Balancer, we use only this interface.

> Balancer should work with the logic of BlockPlacementPolicy
> -----------------------------------------------------------
>                 Key: HDFS-1431
>                 URL: https://issues.apache.org/jira/browse/HDFS-1431
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: balancer
>    Affects Versions: 0.22.0
>            Reporter: Scott Chen
>            Assignee: Scott Chen
>             Fix For: 0.22.0
>         Attachments: HDFS-1431.txt
> Currently Balancer does not obtain information from BlockPlacementPolicy so it can transfer
the blocks without checking with BlockPlacementPolicy.
> This causes the policy break after balancing the cluster.
> There are some new policies proposed in HDFS-1094 and MAPREDUCE-1831 in which the block
placement follows some pattern.
> The pattern can be broken by Balancer.
> I propose that we add the following method in BlockPlacementPolicy:
> {code}
>   abstract public boolean canBeMoved(String fileName, Block block,
>     DatanodeInfo source, DatanodeInfo destination);
> {code}
> And make Balancer use it in
> {code}
>   private boolean isGoodBlockCandidate(Source source,
>       BalancerDatanode target, BalancerBlock block)
> {code}
> What do you think?

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message