hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chen He (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6342) TestBalancerWithNodeGroup.testBalancerWithRackLocality may fail if balancer.id file is huge
Date Tue, 06 May 2014 13:55:21 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990654#comment-13990654
] 

Chen He commented on HDFS-6342:
-------------------------------

Hi [~decster]
1) For the null balancer.id file, I disagree with you. Sometimes, administrators or users
want to know which node runs the balancer. Hostname in the balancer.id file can provide location
information if cluster becomes large.  
2) For the timeout issue, in this patch, I did not change the timeout, it is still 40 seconds.
Average execution time of this method is about 31 seconds (average of 40 times).


> TestBalancerWithNodeGroup.testBalancerWithRackLocality may fail if balancer.id file is
huge
> -------------------------------------------------------------------------------------------
>
>                 Key: HDFS-6342
>                 URL: https://issues.apache.org/jira/browse/HDFS-6342
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Chen He
>            Assignee: Chen He
>         Attachments: HDFS-6342.patch
>
>
> The testBalancerWithRackLocality mehtod is to test balancer moving data blocks with rack
locality consideration. 
> It crates two nodes cluster. One node belongs to rack0nodeGroup0, theother node blongs
to rack1nodeGroup1. In this 2 datanodes minicluster, block size is 10B and total cluster capacity
is 6000B ( 3000B on each datanodes). It create 180 data blocks with replication factor 2.
Then, a node datanode is created (in rack1nodeGroup2) and balancer starts to balancing the
cluster.
> It expects there is only data blocks moving within rack1. After balancer is done, it
assumes the data size on both racks is the same. It will break
> if balancer.id file is huge and there is inter-rack data block moving.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message