hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shahab Yunus <shahab.yu...@gmail.com>
Subject Re: Chaining MapReduce
Date Fri, 21 Aug 2015 17:37:35 GMT
What is the different between the mappers? Is the input data suppose to go
to all mappers or it is dependent on the source data?

Regards,
Shahab

On Fri, Aug 21, 2015 at 1:35 PM, ☼ R Nair (रविशंकर नायर) <
ravishankar.nair@gmail.com> wrote:

> All,
>
> I have three mappers, followed by a reducer. I executed the map reduce
> successfully. The reported output shows that number of mappers executed is
> 1 and number of reducers is also 1. Though number of reducers are correct,
> won't we be getting number of mappers as 3 , since I have three mapper
> classes connected by ChainMapper?
>
> O/P given below (snippet) :-
>
> Job Counters
>         Launched map tasks=1
>         Launched reduce tasks=1
>         Data-local map tasks=1
>         Total time spent by all maps in occupied slots (ms)=8853
>         Total time spent by all reduces in occupied slots (ms)=9900
>         Total time spent by all map tasks (ms)=8853
>         Total time spent by all reduce tasks (ms)=9900
>         Total vcore-seconds taken by all map tasks=8853
>         Total vcore-seconds taken by all reduce tasks=9900
>         Total megabyte-seconds taken by all map tasks=9065472
>         Total megabyte-seconds taken by all reduce tasks=10137600
>
>
> What I guess is, since the output is passing through Context, the internal
> connected mappers are not caught by job counter, am I correct ?
>
> Best, Ravion
>

Mime
View raw message