hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From shanthi k <kshanthi...@gmail.com>
Subject Re: Reduce starts before map completes (at 23%)
Date Fri, 12 Apr 2013 04:57:23 GMT
hello Sai,

As you said Reducer starts only after completion of mapper....... U r
doubting that y reducer is staring at 23% of mapper right? The answer
is,there is a copy phase between mapper and reducer......after mapper as
completed some tasks ,then those tasks are being copied to reducer......by
copying also you can see that reducer % is increasing.......you also notice
that only after mapper has completed 100% reducer completes its job
100%........thanq,Hope u will understand


On Thu, Apr 11, 2013 at 11:15 PM, Sai Sai <saigraph@yahoo.in> wrote:

> I am running the wordcount from hadoop-examples, i am giving as input a
> bunch of test files, i have noticed in the output given below reduce starts
> when the map is at 23%, i was wondering if it is not right that reducers
> will start only after the complete mapping is done which mean when map is
> 100% then i thought the reducers will start. Why r the reducers starting
> when map is still at 23%.
>
> 13/04/11 21:10:32 INFO mapred.JobClient:  map 0% reduce 0%
> 13/04/11 21:10:56 INFO mapred.JobClient:  map 1% reduce 0%
> 13/04/11 21:10:59 INFO mapred.JobClient:  map 2% reduce 0%
> 13/04/11 21:11:02 INFO mapred.JobClient:  map 3% reduce 0%
> 13/04/11 21:11:05 INFO mapred.JobClient:  map 4% reduce 0%
> 13/04/11 21:11:08 INFO mapred.JobClient:  map 6% reduce 0%
> 13/04/11 21:11:11 INFO mapred.JobClient:  map 7% reduce 0%
> 13/04/11 21:11:17 INFO mapred.JobClient:  map 8% reduce 0%
> 13/04/11 21:11:23 INFO mapred.JobClient:  map 10% reduce 0%
> 13/04/11 21:11:26 INFO mapred.JobClient:  map 12% reduce 0%
> 13/04/11 21:11:32 INFO mapred.JobClient:  map 14% reduce 0%
> 13/04/11 21:11:44 INFO mapred.JobClient:  map 23% reduce 0%
> 13/04/11 21:11:50 INFO mapred.JobClient:  map 23% reduce 1%
> 13/04/11 21:11:53 INFO mapred.JobClient:  map 33% reduce 7%
> 13/04/11 21:12:02 INFO mapred.JobClient:  map 42% reduce 7%
>
> Please pour some light.
> Thanks
> Sai
>

Mime
View raw message