hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Deyaa Adranale <deyaa.adran...@iais.fraunhofer.de>
Subject Re: How to control the map and reduce step sequentially
Date Mon, 28 Jul 2008 10:29:43 GMT
As far as I know, the reducer has three tasks: fetching results of 
mappers, sorting the results, and calling the reduce function.
when some mappers finish their execution, the reducer starts by fetching 
their results to save time.
neither sorting nor calling the reduce function could start before all 
the mappers have finished and all their results are available locally.

I don't know whether you can prevent copying mappers results before all 
mappers finish. Anyway, it would be meaningless.

hope that helped


??? wrote:
> Dear All,
> When i using Hadoop, I noticed that the reducer step is started immediately
> when the mappers are still running. According to my project requirement, the
> reducer step should not start until all the mappers finish their execution.
> Anybody knows how to use some Hadoop API to achieve this? When all the
> mappers finish their process, then the reducer is started.
> Thanks

View raw message