Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
MIME-Version: 1.0
Date: Fri, 21 Aug 2015 10:35:17 -0700
Message-ID: 
 <CAB1GMv=eKXfKzZtPaDc0_1qXKx50ZWQSqtMKpzDUh5Bn=mOeMg@mail.gmail.com>
Subject: Chaining MapReduce
From: 
 =?UTF-8?B?4pi8IFIgTmFpciAo4KSw4KS14KS/4KS24KSC4KSV4KSwIOCkqOCkvuCkr+CksCk=?=
 <ravishankar.nair@gmail.com>
To: user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=001a11391864f64576051dd5b009

--001a11391864f64576051dd5b009
Content-Type: text/plain; charset=UTF-8

All,

I have three mappers, followed by a reducer. I executed the map reduce
successfully. The reported output shows that number of mappers executed is
1 and number of reducers is also 1. Though number of reducers are correct,
won't we be getting number of mappers as 3 , since I have three mapper
classes connected by ChainMapper?

O/P given below (snippet) :-

Job Counters
        Launched map tasks=1
        Launched reduce tasks=1
        Data-local map tasks=1
        Total time spent by all maps in occupied slots (ms)=8853
        Total time spent by all reduces in occupied slots (ms)=9900
        Total time spent by all map tasks (ms)=8853
        Total time spent by all reduce tasks (ms)=9900
        Total vcore-seconds taken by all map tasks=8853
        Total vcore-seconds taken by all reduce tasks=9900
        Total megabyte-seconds taken by all map tasks=9065472
        Total megabyte-seconds taken by all reduce tasks=10137600


What I guess is, since the output is passing through Context, the internal
connected mappers are not caught by job counter, am I correct ?

Best, Ravion

--001a11391864f64576051dd5b009
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div><div>All,<br><br></div>I have three mappers, followed=
 by a reducer. I executed the map reduce successfully. The reported output =
shows that number of mappers executed is 1 and number of reducers is also 1=
. Though number of reducers are correct, won&#39;t we be getting number of =
mappers as 3 , since I have three mapper classes connected by ChainMapper?<=
br><br></div><div>O/P given below (snippet) :-<br><br>Job Counters <br>=C2=
=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 Launched map tasks=3D1<br>=C2=A0=C2=A0=
=C2=A0 =C2=A0=C2=A0=C2=A0 Launched reduce tasks=3D1<br>=C2=A0=C2=A0=C2=A0 =
=C2=A0=C2=A0=C2=A0 Data-local map tasks=3D1<br>=C2=A0=C2=A0=C2=A0 =C2=A0=C2=
=A0=C2=A0 Total time spent by all maps in occupied slots (ms)=3D8853<br>=C2=
=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 Total time spent by all reduces in occup=
ied slots (ms)=3D9900<br>=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 Total time s=
pent by all map tasks (ms)=3D8853<br>=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =
Total time spent by all reduce tasks (ms)=3D9900<br>=C2=A0=C2=A0=C2=A0 =C2=
=A0=C2=A0=C2=A0 Total vcore-seconds taken by all map tasks=3D8853<br>=C2=A0=
=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 Total vcore-seconds taken by all reduce tas=
ks=3D9900<br>=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 Total megabyte-seconds t=
aken by all map tasks=3D9065472<br>=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 To=
tal megabyte-seconds taken by all reduce tasks=3D10137600<br><br><br></div>=
<div>What I guess is, since the output is passing through Context, the inte=
rnal connected mappers are not caught by job counter, am I correct ?<br></d=
iv><div><br></div>Best, Ravion<br></div>

--001a11391864f64576051dd5b009--