Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Tue, 6 May 2014 02:31:17 +0000 (UTC)
From: "Binglin Chang (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12712332.1399303273665.233968.1399343477470@arcas>
In-Reply-To: <JIRA.12712332.1399303273665@arcas>
References: <JIRA.12712332.1399303273665@arcas>
Subject: [jira] [Commented] (HDFS-6342)
 TestBalancerWithNodeGroup.testBalancerWithRackLocality may fail if
 balancer.id file is huge
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-6342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990211#comment-13990211 ] 

Binglin Chang commented on HDFS-6342:
-------------------------------------

As for the fix, I see the need to write a balancer id file, but fill it with hostname doesn't seem to be necessary(cause it is never used anywhere), so if we can modify balancer, write the balancer file but don't write any content, it should not have side effects to balancer and test check code, and we may skip timeout(need to confirm)

> TestBalancerWithNodeGroup.testBalancerWithRackLocality may fail if balancer.id file is huge
> -------------------------------------------------------------------------------------------
>
>                 Key: HDFS-6342
>                 URL: https://issues.apache.org/jira/browse/HDFS-6342
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Chen He
>            Assignee: Chen He
>         Attachments: HDFS-6342.patch
>
>
> The testBalancerWithRackLocality mehtod is to test balancer moving data blocks with rack locality consideration. 
> It crates two nodes cluster. One node belongs to rack0nodeGroup0, theother node blongs to rack1nodeGroup1. In this 2 datanodes minicluster, block size is 10B and total cluster capacity is 6000B ( 3000B on each datanodes). It create 180 data blocks with replication factor 2. Then, a node datanode is created (in rack1nodeGroup2) and balancer starts to balancing the cluster.
> It expects there is only data blocks moving within rack1. After balancer is done, it assumes the data size on both racks is the same. It will break
> if balancer.id file is huge and there is inter-rack data block moving.


--
This message was sent by Atlassian JIRA
(v6.2#6252)