hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ☼ R Nair (रविशंकर नायर) <ravishankar.n...@gmail.com>
Subject Chaining MapReduce
Date Fri, 21 Aug 2015 17:35:17 GMT
All,

I have three mappers, followed by a reducer. I executed the map reduce
successfully. The reported output shows that number of mappers executed is
1 and number of reducers is also 1. Though number of reducers are correct,
won't we be getting number of mappers as 3 , since I have three mapper
classes connected by ChainMapper?

O/P given below (snippet) :-

Job Counters
        Launched map tasks=1
        Launched reduce tasks=1
        Data-local map tasks=1
        Total time spent by all maps in occupied slots (ms)=8853
        Total time spent by all reduces in occupied slots (ms)=9900
        Total time spent by all map tasks (ms)=8853
        Total time spent by all reduce tasks (ms)=9900
        Total vcore-seconds taken by all map tasks=8853
        Total vcore-seconds taken by all reduce tasks=9900
        Total megabyte-seconds taken by all map tasks=9065472
        Total megabyte-seconds taken by all reduce tasks=10137600


What I guess is, since the output is passing through Context, the internal
connected mappers are not caught by job counter, am I correct ?

Best, Ravion

Mime
View raw message