Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CAB1GMv=eKXfKzZtPaDc0_1qXKx50ZWQSqtMKpzDUh5Bn=mOeMg@mail.gmail.com>
References: 
 <CAB1GMv=eKXfKzZtPaDc0_1qXKx50ZWQSqtMKpzDUh5Bn=mOeMg@mail.gmail.com>
Date: Fri, 21 Aug 2015 13:37:35 -0400
Message-ID: 
 <CAEo-6+QrKg-+sEWywEN_bis_vCKaKMCM6x+e74o8BybsMkT=jg@mail.gmail.com>
Subject: Re: Chaining MapReduce
From: Shahab Yunus <shahab.yunus@gmail.com>
To: "user@hadoop.apache.org" <user@hadoop.apache.org>
Content-Type: multipart/alternative; boundary=001a113925362dda44051dd5b970

--001a113925362dda44051dd5b970
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

What is the different between the mappers? Is the input data suppose to go
to all mappers or it is dependent on the source data?

Regards,
Shahab

On Fri, Aug 21, 2015 at 1:35 PM, =E2=98=BC R Nair (=E0=A4=B0=E0=A4=B5=E0=A4=
=BF=E0=A4=B6=E0=A4=82=E0=A4=95=E0=A4=B0 =E0=A4=A8=E0=A4=BE=E0=A4=AF=E0=A4=
=B0) <
ravishankar.nair@gmail.com> wrote:

> All,
>
> I have three mappers, followed by a reducer. I executed the map reduce
> successfully. The reported output shows that number of mappers executed i=
s
> 1 and number of reducers is also 1. Though number of reducers are correct=
,
> won't we be getting number of mappers as 3 , since I have three mapper
> classes connected by ChainMapper?
>
> O/P given below (snippet) :-
>
> Job Counters
>         Launched map tasks=3D1
>         Launched reduce tasks=3D1
>         Data-local map tasks=3D1
>         Total time spent by all maps in occupied slots (ms)=3D8853
>         Total time spent by all reduces in occupied slots (ms)=3D9900
>         Total time spent by all map tasks (ms)=3D8853
>         Total time spent by all reduce tasks (ms)=3D9900
>         Total vcore-seconds taken by all map tasks=3D8853
>         Total vcore-seconds taken by all reduce tasks=3D9900
>         Total megabyte-seconds taken by all map tasks=3D9065472
>         Total megabyte-seconds taken by all reduce tasks=3D10137600
>
>
> What I guess is, since the output is passing through Context, the interna=
l
> connected mappers are not caught by job counter, am I correct ?
>
> Best, Ravion
>

--001a113925362dda44051dd5b970
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">What is the different between the mappers? Is the input da=
ta suppose to go to all mappers or it is dependent on the source data?<div>=
<br></div><div>Regards,</div><div>Shahab</div></div><div class=3D"gmail_ext=
ra"><br><div class=3D"gmail_quote">On Fri, Aug 21, 2015 at 1:35 PM, =E2=98=
=BC R Nair (=E0=A4=B0=E0=A4=B5=E0=A4=BF=E0=A4=B6=E0=A4=82=E0=A4=95=E0=A4=B0=
 =E0=A4=A8=E0=A4=BE=E0=A4=AF=E0=A4=B0) <span dir=3D"ltr">&lt;<a href=3D"mai=
lto:ravishankar.nair@gmail.com" target=3D"_blank">ravishankar.nair@gmail.co=
m</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margi=
n:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr">=
<div><div>All,<br><br></div>I have three mappers, followed by a reducer. I =
executed the map reduce successfully. The reported output shows that number=
 of mappers executed is 1 and number of reducers is also 1. Though number o=
f reducers are correct, won&#39;t we be getting number of mappers as 3 , si=
nce I have three mapper classes connected by ChainMapper?<br><br></div><div=
>O/P given below (snippet) :-<br><br>Job Counters <br>=C2=A0=C2=A0=C2=A0 =
=C2=A0=C2=A0=C2=A0 Launched map tasks=3D1<br>=C2=A0=C2=A0=C2=A0 =C2=A0=C2=
=A0=C2=A0 Launched reduce tasks=3D1<br>=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=
=A0 Data-local map tasks=3D1<br>=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 Total=
 time spent by all maps in occupied slots (ms)=3D8853<br>=C2=A0=C2=A0=C2=A0=
 =C2=A0=C2=A0=C2=A0 Total time spent by all reduces in occupied slots (ms)=
=3D9900<br>=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 Total time spent by all ma=
p tasks (ms)=3D8853<br>=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 Total time spe=
nt by all reduce tasks (ms)=3D9900<br>=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0=
 Total vcore-seconds taken by all map tasks=3D8853<br>=C2=A0=C2=A0=C2=A0 =
=C2=A0=C2=A0=C2=A0 Total vcore-seconds taken by all reduce tasks=3D9900<br>=
=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 Total megabyte-seconds taken by all m=
ap tasks=3D9065472<br>=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 Total megabyte-=
seconds taken by all reduce tasks=3D10137600<br><br><br></div><div>What I g=
uess is, since the output is passing through Context, the internal connecte=
d mappers are not caught by job counter, am I correct ?<br></div><div><br><=
/div>Best, Ravion<br></div>
</blockquote></div><br></div>

--001a113925362dda44051dd5b970--