hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Senthil Kumar <senthilec...@gmail.com>
Subject Re: HDFS Balancer - Recommended Threshold Value
Date Wed, 10 Aug 2016 13:31:56 GMT
Hadoop Version :

*Hadoop 2.4.1*
--Senthil

On Wed, Aug 10, 2016 at 6:58 PM, Zhe Zhang <zhe.zhang.research@gmail.com>
wrote:

> Which version of Hadoop is your cluster / Balancer running?
>
> On Wed, Aug 10, 2016 at 6:27 AM, Senthil Kumar <senthilec566@gmail.com>
> wrote:
>
>> Hi Team ,  We are running big cluster ( 3000 nodes cluster ) , many time
>> we
>> are hitting  Median Increasing to 99.99 % ( 80 % of the DN's ) .  Balancer
>> is running all time in cluster ..But still  median is not coming down i.e
>> <
>> 90 % ..
>>
>> Here is how i start balancer ?
>> /apache/hadoop/sbin/start-balancer.sh
>> -Ddfs.balance.bandwidthPerSec=104857600  *-threshold  30*
>>
>> What the recommended value for thershold ??  Is there any way to pass
>> param
>> only to move blocks from Over Utilized ( 98-100%) to under utilized ?
>>
>>
>> Pls advise!
>>
>>
>>
>>
>> Regards,
>> Senthil
>>
>
>
>
> --
> Zhe Zhang
> Apache Hadoop Commiter
> http://zhe-thoughts.github.io/about/ | @oldcap
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message