hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley" <omal...@apache.org>
Subject Re: Doubt regarding use of HTTP during Shuffle phase
Date Wed, 27 Jun 2012 16:40:27 GMT
On Wed, Jun 27, 2012 at 9:33 AM, Pavan Kulkarni <pavan.baburao@gmail.com>wrote:

Why is HTTP used during the Shuffle phase
> instead of just creating a Hardlink?

The reduces run on a separate machine that the map. Hadoop doesn't use a
shared NFS file system between the machines, because it doesn't scale.

-- Owen

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message