hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From daemeon reiydelle <daeme...@gmail.com>
Subject Re:
Date Tue, 11 Nov 2014 23:34:43 GMT
I observed what I thought was a similar problem but found that I was
actually hitting the physical disks much harder/more efficiently ... I
found that I had some spindles with pretty deep queue depths. When I sat
back and actually measured, I found that my throughput had substantially
increased, moving the bottleneck to the disks. What does iostats tell you?









*.......“The race is not to the swift,nor the battle to the strong,but to
those who can see it coming and jump aside.” - Hunter ThompsonDaemeon C.M.
ReiydelleUSA (+1) 415.501.0198London (+44) (0) 20 8144 9872*

On Tue, Nov 11, 2014 at 2:43 PM, Premal Shah <premal.j.shah@gmail.com>
wrote:

> Hi,
> We recently upgraded from 1.0.4 to 1.2.1 and now have a ton of DFS
> balancing to do.
> On 1.0.4, when I set the bandwidth to 1 or 10 Gbps, the datanodes got
> working and were using a ton of network to get the nodes balanced. However,
> looks like 1.2.1 is not honoring the setBandWidth setting, no matter how
> high I set it to be. The datanodes are using only around 30Mbps. Is there a
> way to speed up the balancing?
>
> --
> Regards,
> Premal Shah.
>

Mime
View raw message