hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chang Li (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8818) Allow Balancer to run faster
Date Tue, 28 Jul 2015 16:40:05 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14644620#comment-14644620
] 

Chang Li commented on HDFS-8818:
--------------------------------

[~szetszwo] thanks for the patch. One thing I am concerned is the change
{code}
-      return srcBlocks.size() < SOURCE_BLOCKS_MIN_SIZE && blocksToReceive >
0;
+      return blocksToReceive > 0;
{code}
now that dispatcher will keep fetching more blocks from namenode every iteration, but namenode
is likely to return very same list of blocks since the block moving is not that fast and namenode
can't know the blocks just moved instantly. This could increase useless load on namenode.


> Allow Balancer to run faster
> ----------------------------
>
>                 Key: HDFS-8818
>                 URL: https://issues.apache.org/jira/browse/HDFS-8818
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: balancer & mover
>            Reporter: Tsz Wo Nicholas Sze
>            Assignee: Tsz Wo Nicholas Sze
>         Attachments: h8818_20150723.patch, h8818_20150727.patch
>
>
> The original design of Balancer is intentionally to make it run slowly so that the balancing
activities won't affect the normal cluster activities and the running jobs.
> There are new use case that cluster admin may choose to balance the cluster when the
cluster load is low, or in a maintain window.  So that we should have an option to allow Balancer
to run faster.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message