hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinod Kumar Vavilapalli <vino...@hortonworks.com>
Subject Re: How reduce tasks know which partition they should read?
Date Mon, 09 Mar 2015 22:21:14 GMT

The reducers(Fetcher.java) simply ask the Shuffle Service (ShuffleHandler.java) to give them
output corresponding to a specific map. The partitioning detail is hidden from the reducers.


On Mar 9, 2015, at 7:56 AM, xeonmailinglist-gmail <xeonmailinglist@gmail.com> wrote:

> Hi,
> I am looking to the Yarn mapreduce internals to try to understand how reduce tasks know
which partition of the map output they should read. Even, when they re-execute after a crash?
> I am also looking to the mapreduce source code. Is there any class that I should look
to try to understand this question?
> Any help?
> Thanks
> -- 
> --

View raw message