hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Armstrong <john.armstr...@ccri.com>
Subject Re: Why inter-rack communication in mapreduce slow?
Date Mon, 06 Jun 2011 13:21:51 GMT
On Mon, 06 Jun 2011 09:18:45 -0400, <darren@ontrenet.com> wrote:
> I never understood how hadoop can throttle an inter-rack fiber switch.
> Its supposed to operate on the principle of move-the-code to the data
> because of the I/O cost of moving the data, right?

But what happens when a reducer on rack A gets most of its input from
mappers on rack A, but needs a serious chunk of data from mappers on racks,
B, C, D...

View raw message