hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo Nicholas Sze (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8875) Optimize the wait time in Balancer for federation scenario
Date Tue, 11 Aug 2015 21:26:53 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14692284#comment-14692284
] 

Tsz Wo Nicholas Sze commented on HDFS-8875:
-------------------------------------------

Balancer will exit if one of the NNs succeeds or throws exception.  See if you also want to
fix it here.

> Optimize the wait time in Balancer for federation scenario
> ----------------------------------------------------------
>
>                 Key: HDFS-8875
>                 URL: https://issues.apache.org/jira/browse/HDFS-8875
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Ming Ma
>            Assignee: Chris Trezzo
>
> Balancer has wait time between two consecutive iterations. That is to give some time
for block movement to be fully committed ( return from replaceBlock doesn't mean the NN's
blockmap has been updated and the block has been invalidated on the source node.).
> This wait time could be 23 seconds if {{dfs.heartbeat.interval}} is set to 10 and {{dfs.namenode.replication.interval}}
is to 3. In the case of federation, given we iterate through all namespaces in each iteration,
this wait time becomes unnecessary as while balancer is processing the next namespace, it
gives the previous namespace it just finished time to commit.
> In addition, Balancer calls {{Collections.shuffle(connectors);}} It doesn't seem necessary.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message