hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mori Bellamy <mbell...@apple.com>
Subject Re: When does reducer read mapper's intermediate result?
Date Mon, 14 Jul 2008 22:59:10 GMT
i'm pretty sure that the reducer waits for all of the map tasks'  
output to be written to HDFS (or else i nee no use for the Combiner  
class).  i'm not sure about your second question though. my gut tells  
me "no"

On Jul 14, 2008, at 3:50 PM, Kevin wrote:

> Hi, there,
> I am interested in the implementation details of hadoop mapred. In
> particular, does the reducer wait till a map task ends and then fetch
> the output (key-value pairs)? If so, is the very file produced by a
> mapper for the reducer sorted before reducer gets it? (which means
> that the reducer only needs to do merge sort when it gets all the
> intermediate files from different mappers).
> Best,
> -Kevin

View raw message