Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
  s=s1024; d=yahoo.in;
  h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type;
  b=DNzklTzt8LTRHgm2LR5B1mycN3NIqfyjHTUPMtvlFs99j59+WzIcNzlY7VRZU1hZ1vUNV5EtI8KT1rdzEAXe/9VTSC/a0X/I/aqS/IkvztbwLxKI1BT4uI1jvyqJZb8+i/IKixf/g3y3Ev+oShGkPz9otyCt+lSTt/0DURxetOc=;
References: <1364377874.13753.YahooMailNeo@web194703.mail.sg3.yahoo.com>
  <CADfVb56UdbZCsDd=K7+ruYcJ_veVYEQVAdKouYJChT32Uvv5+g@mail.gmail.com>
  <1364577771.12724.YahooMailNeo@web194704.mail.sg3.yahoo.com>
  <1364719534.91394.YahooMailNeo@web194703.mail.sg3.yahoo.com>
  <1365042870.89547.YahooMailNeo@web194702.mail.sg3.yahoo.com>
  <1365740112.75877.YahooMailNeo@web190702.mail.sg3.yahoo.com>
  <CAH4HbuEQPR=ec6P_km1bO19jB4sTPF_YQdFKa=ueZPNmWDsU7g@mail.gmail.com>
 <578478094-1365742754-cardhu_decombobulator_blackberry.rim.net-2001349931-@b16.c6.bise7.blackberry>
 <1365754060.82642.YahooMailNeo@web190701.mail.sg3.yahoo.com>
Message-ID: <1365817528.61054.YahooMailNeo@web190703.mail.sg3.yahoo.com>
Date: Sat, 13 Apr 2013 09:45:28 +0800 (SGT)
From: Sai Sai <saigraph@yahoo.in>
Reply-To: Sai Sai <saigraph@yahoo.in>
Subject: Re: 100K Maps scenario
To: "user@hadoop.apache.org" <user@hadoop.apache.org>
In-Reply-To: <1365754060.82642.YahooMailNeo@web190701.mail.sg3.yahoo.com>
MIME-Version: 1.0
Content-Type: multipart/alternative;
 boundary="-300775223-376415299-1365817528=:61054"

---300775223-376415299-1365817528=:61054
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable

=0A=0AJust a follow up to see if anyone can shed some light on this:=0AMy u=
nderstanding is that each block after getting replicated 3 times, a map tas=
k is run on each of the replica in parallel.=0AThe thing i am trying to dou=
ble verify is in a scenario where a file is split into 10K or 100K or more =
blocks it will result in atleast 300K Map tasks being performed and this lo=
oks like an overkill from a performance or just a logical perspective.=A0=
=0AWill appreciate any thoughts on this.=0AThanks=0ASai=0A=0A______________=
__________________=0A From: Sai Sai <saigraph@yahoo.in>=0ATo: "user@hadoop.=
apache.org" <user@hadoop.apache.org>; Sai Sai <saigraph@yahoo.in> =0ASent: =
Friday, 12 April 2013 1:37 PM=0ASubject: Re: Does a Map task run 3 times on=
 3 TTs or just once=0A =0A=0A=0AJust wondering if it is right to assume tha=
t a Map task is run 3 times on 3 different TTs in parallel and whoever comp=
letes processing the task first that output is picked up and written to int=
ermediate location.=0AOr is it true that a map task even though its data is=
 replicated 3 times will run only once and other 2 will be on the stand by =
just incase this fails the second one will run followed by 3rd one if the 2=
nd Mapper fails.=0APlesae pour some light.=0AThanks=0ASai
---300775223-376415299-1365817528=:61054
Content-Type: text/html; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable

<html><body><div style=3D"color:#000; background-color:#fff; font-family:ti=
mes new roman, new york, times, serif;font-size:12pt"><div><span><br></span=
></div><div>Just a follow up to see if anyone can shed some light on this:<=
/div><div>My understanding is that each block after getting replicated 3 ti=
mes, a map task is run on each of the replica in parallel.</div><div>The th=
ing i am trying to double verify is in a scenario where a file is split int=
o 10K or 100K or more blocks it will result in atleast 300K Map tasks being=
 performed and this looks like an overkill from a performance or just a log=
ical perspective.&nbsp;</div><div>Will appreciate any thoughts on this.</di=
v><div>Thanks</div><div>Sai</div>  <div style=3D"font-family: 'times new ro=
man', 'new york', times, serif; font-size: 12pt; "> <div style=3D"font-fami=
ly: 'times new roman', 'new york', times, serif; font-size: 12pt; "> <div d=
ir=3D"ltr"> <hr size=3D"1">  <font size=3D"2" face=3D"Arial"> <b><span
 style=3D"font-weight:bold;">From:</span></b> Sai Sai &lt;saigraph@yahoo.in=
&gt;<br> <b><span style=3D"font-weight: bold;">To:</span></b> "user@hadoop.=
apache.org" &lt;user@hadoop.apache.org&gt;; Sai Sai &lt;saigraph@yahoo.in&g=
t; <br> <b><span style=3D"font-weight: bold;">Sent:</span></b> Friday, 12 A=
pril 2013 1:37 PM<br> <b><span style=3D"font-weight: bold;">Subject:</span>=
</b> Re: Does a Map task run 3 times on 3 TTs or just once<br> </font> </di=
v> <div class=3D"y_msg_container"><br><meta http-equiv=3D"x-dns-prefetch-co=
ntrol" content=3D"off"><div id=3D"yiv1056091979"><div><div style=3D"color: =
rgb(0, 0, 0); background-color: rgb(255, 255, 255); font-family: 'times new=
 roman', 'new york', times, serif; font-size: 12pt; "><div>Just wondering i=
f it is right to assume that a Map task is run 3 times on 3 different TTs i=
n parallel and whoever completes processing the task first that output is p=
icked up and written to intermediate location.</div><div>Or is it true that=
 a map task
 even though its data is replicated 3 times will run only once and other 2 =
will be on the stand by just incase this fails the second one will run foll=
owed by 3rd one if the 2nd Mapper fails.</div><div>Plesae pour some light.<=
/div><div>Thanks</div><div>Sai</div>  </div></div></div><meta http-equiv=3D=
"x-dns-prefetch-control" content=3D"on"><br><br></div> </div> </div>  </div=
></body></html>
---300775223-376415299-1365817528=:61054--