hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rakesh R <rake...@huawei.com>
Subject RE: issue about cluster balance
Date Tue, 06 May 2014 04:38:35 GMT
Could you give more details like,

-          Could you convert 7% to the total amount of moved data in MBs.

-          Also, could you tell me 7% data movement per DN ?

-          What values showing for the ‘over-utilized’, ‘above-average’, ‘below-average’,
‘below-average’ nodes. Balancer will do the pairing based on these values.

-          Please tell me the cluster topology - SAME_NODE_GROUP, SAME_RACK. Basically this
will matters when choosing the sourceNode vs balancerNode pairs as well as the proxy source.

Did you see all the DNs are getting utilized for the block movement.

-          Any exceptions occurred when block movement

-          How many iterations played in these hours


From: ch huang [mailto:justlooks@gmail.com]
Sent: 06 May 2014 06:10
To: user@hadoop.apache.org
Subject: issue about cluster balance

                 i have a 5-node hadoop cluster,and yesterday i add 5 new box into my cluster,after
that i start balance task,but it move only 7% data to new node in 20 hour , and i already
set dfs.datanode.balance.bandwidthPerSec 10M ,and the threshold is 10%,why the balance task
take long time ?
View raw message