hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Binglin Chang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6342) TestBalancerWithNodeGroup.testBalancerWithRackLocality may fail if balancer.id file is huge
Date Tue, 06 May 2014 02:31:17 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990211#comment-13990211
] 

Binglin Chang commented on HDFS-6342:
-------------------------------------

As for the fix, I see the need to write a balancer id file, but fill it with hostname doesn't
seem to be necessary(cause it is never used anywhere), so if we can modify balancer, write
the balancer file but don't write any content, it should not have side effects to balancer
and test check code, and we may skip timeout(need to confirm)

> TestBalancerWithNodeGroup.testBalancerWithRackLocality may fail if balancer.id file is
huge
> -------------------------------------------------------------------------------------------
>
>                 Key: HDFS-6342
>                 URL: https://issues.apache.org/jira/browse/HDFS-6342
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Chen He
>            Assignee: Chen He
>         Attachments: HDFS-6342.patch
>
>
> The testBalancerWithRackLocality mehtod is to test balancer moving data blocks with rack
locality consideration. 
> It crates two nodes cluster. One node belongs to rack0nodeGroup0, theother node blongs
to rack1nodeGroup1. In this 2 datanodes minicluster, block size is 10B and total cluster capacity
is 6000B ( 3000B on each datanodes). It create 180 data blocks with replication factor 2.
Then, a node datanode is created (in rack1nodeGroup2) and balancer starts to balancing the
cluster.
> It expects there is only data blocks moving within rack1. After balancer is done, it
assumes the data size on both racks is the same. It will break
> if balancer.id file is huge and there is inter-rack data block moving.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message