hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yeqi <ye...@huawei.com>
Subject 答复: HDFS Balancer - Recommended Threshold Value
Date Thu, 11 Aug 2016 00:39:23 GMT
Try to set a higher number of DFS_DATANODE_BALANCE_MAX_NUM_CONCURRENT_MOVES_KEY will help.
Default is 5.
It's won't help much if u only use bandwidthPerSec and threshold.

-----邮件原件-----
发件人: Senthil Kumar [mailto:senthilec566@gmail.com] 
发送时间: 2016年8月10日 21:28
收件人: hdfs-dev@hadoop.apache.org; senthilec566@apache.org
主题: HDFS Balancer - Recommended Threshold Value

Hi Team ,  We are running big cluster ( 3000 nodes cluster ) , many time we are hitting  Median
Increasing to 99.99 % ( 80 % of the DN's ) .  Balancer is running all time in cluster ..But
still  median is not coming down i.e <
90 % ..

Here is how i start balancer ?
/apache/hadoop/sbin/start-balancer.sh
-Ddfs.balance.bandwidthPerSec=104857600  *-threshold  30*

What the recommended value for thershold ??  Is there any way to pass param only to move blocks
from Over Utilized ( 98-100%) to under utilized ?


Pls advise!




Regards,
Senthil
Mime
View raw message