hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ☼ R Nair (रविशंकर नायर) <ravishankar.n...@gmail.com>
Subject Re: Chaining MapReduce
Date Sat, 22 Aug 2015 06:02:00 GMT
Hi ,

The mappers depend on source data only. But data definitely is going
through all mappers, so I should get number of map jpbs as my output right?
Instead I am getting only one.

Thanks and regards,
Ravion

On Fri, Aug 21, 2015 at 1:35 PM, ☼ R Nair (रविशंकर नायर) <
ravishankar.nair@gmail.com> wrote:

> All,
>
> I have three mappers, followed by a reducer. I executed the map reduce
> successfully. The reported output shows that number of mappers executed is
> 1 and number of reducers is also 1. Though number of reducers are correct,
> won't we be getting number of mappers as 3 , since I have three mapper
> classes connected by ChainMapper?
>
> O/P given below (snippet) :-
>
> Job Counters
>         Launched map tasks=1
>         Launched reduce tasks=1
>         Data-local map tasks=1
>         Total time spent by all maps in occupied slots (ms)=8853
>         Total time spent by all reduces in occupied slots (ms)=9900
>         Total time spent by all map tasks (ms)=8853
>         Total time spent by all reduce tasks (ms)=9900
>         Total vcore-seconds taken by all map tasks=8853
>         Total vcore-seconds taken by all reduce tasks=9900
>         Total megabyte-seconds taken by all map tasks=9065472
>         Total megabyte-seconds taken by all reduce tasks=10137600
>
>
> What I guess is, since the output is passing through Context, the internal
> connected mappers are not caught by job counter, am I correct ?
>
> Best, Ravion
>

Mime
View raw message