hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Howell <dehow...@gmail.com>
Subject Re: slow transfer rates w/ HDFS balancer
Date Thu, 08 Oct 2009 19:21:57 GMT
Awesome, that did the trick.

Thanks!

On Thu, Oct 8, 2009 at 2:06 PM, Allen Wittenauer
<awittenauer@linkedin.com> wrote:
>
>
>
> On 10/8/09 12:03 PM, "David Howell" <dehowell@gmail.com> wrote:
>
>> I just added a couple of new node to my cluster today, and ran the
>> balancer to get some data onto them. With scp, the transfer rate
>> between these machines is in the neighborhood of 50 MB/s... but the
>> transfers during the balancing cap out at 1 MB/s.
>>
>> Is this to be expected, and if not, does anyone have any ideas of
>> where to start looking for a problem?
>
> Balancer was intended to run as a background job so that the grid isn't
> limping while the balance is going on.  But this is a tunable: change the
> dfs.balance.bandwidthPerSec setting.
>
>

Mime
View raw message