Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of jayunit100@gmail.com designates
 209.85.212.47 as permitted sender)
References: <1369678211.23053.YahooMailNeo@web163002.mail.bf1.yahoo.com>
Mime-Version: 1.0 (1.0)
In-Reply-To: <1369678211.23053.YahooMailNeo@web163002.mail.bf1.yahoo.com>
Content-Type: multipart/alternative;
	boundary=Apple-Mail-1AA78B01-3657-4F6C-B704-5124FAD1D9D0
Content-Transfer-Encoding: 7bit
Message-Id: <E907B541-F1CB-4DAC-88BC-CE8171D8F37D@gmail.com>
Cc: "user@hadoop.apache.org" <user@hadoop.apache.org>
From: Jay Vyas <jayunit100@gmail.com>
Subject: Re: understanding souce code structure
Date: Mon, 27 May 2013 14:22:42 -0400
To: "user@hadoop.apache.org" <user@hadoop.apache.org>


--Apple-Mail-1AA78B01-3657-4F6C-B704-5124FAD1D9D0
Content-Type: text/plain;
	charset=us-ascii
Content-Transfer-Encoding: quoted-printable

Hi!  a few weeks ago I had the same question... Tried a first iteration at d=
ocumenting this by going through the classes starting with key/value pairs i=
n the blog post below. =20

http://jayunit100.blogspot.com/2013/04/the-kv-pair-salmon-run-in-mapreduce-h=
dfs.html

Note it's not perfect yet but I think it should provide some insight into th=
ings.  The lynch pin of it all is the DFSOutputStream and the DataStreamer c=
lasses.   Anyways... Feel free to borrow the contents and roll your own , or=
 comment on it & leave some feedback,or let me know if anything is missing. =
 =20

Definetly would be awesome to have a rock solid view of the full write path.=


On May 27, 2013, at 2:10 PM, Mahmood Naderan <nt_mahmood@yahoo.com> wrote:

> Hello
>=20
> I am trying to understand the source of of hadoop especially the HDFS. I w=
ant to know where should I look exactly in the source code about how HDFS di=
stributes the data. Also how the map reduce engine tries to read the data.=20=

>=20
>=20
> Any hint regarding the location of those in the source code is appreciated=
.
> =20
> Regards,
> Mahmood

--Apple-Mail-1AA78B01-3657-4F6C-B704-5124FAD1D9D0
Content-Type: text/html;
	charset=utf-8
Content-Transfer-Encoding: quoted-printable

<html><head><meta http-equiv=3D"content-type" content=3D"text/html; charset=3D=
utf-8"></head><body dir=3D"auto"><div>Hi! &nbsp;a few weeks ago I had the sa=
me question... Tried a first iteration at documenting this by going through t=
he classes starting with key/value pairs in the blog post below. &nbsp;</div=
><div><br></div><div><div style=3D"-webkit-tap-highlight-color: rgba(26, 26,=
 26, 0.296875); -webkit-composition-fill-color: rgba(175, 192, 227, 0.230469=
); -webkit-composition-frame-color: rgba(77, 128, 180, 0.230469); "><a href=3D=
"http://jayunit100.blogspot.com/2013/04/the-kv-pair-salmon-run-in-mapreduce-=
hdfs.html">http://jayunit100.blogspot.com/2013/04/the-kv-pair-salmon-run-in-=
mapreduce-hdfs.html</a><br></div></div><div><br></div><div>Note it's not per=
fect yet but I think it should provide some insight into things. &nbsp;The l=
ynch pin of it all is the DFSOutputStream and the DataStreamer classes. &nbs=
p;&nbsp;Anyways... Feel free to borrow the contents and roll your own , or c=
omment on it &amp; leave some feedback,or let me know if anything is missing=
. &nbsp;&nbsp;</div><div><br></div><div>Definetly would be awesome to have a=
 rock solid view of the full write path.</div><div><br></div><div>On May 27,=
 2013, at 2:10 PM, Mahmood Naderan &lt;<a href=3D"mailto:nt_mahmood@yahoo.co=
m">nt_mahmood@yahoo.com</a>&gt; wrote:<br><br></div><blockquote type=3D"cite=
"><div><div style=3D"color:#000; background-color:#fff; font-family:Courier N=
ew, courier, monaco, monospace, sans-serif;font-size:12pt"><div><span style=3D=
"font-family: arial,helvetica,sans-serif;">Hello<br><br>I am trying to under=
stand the source of of hadoop especially the HDFS. I want to know where shou=
ld I look exactly in the source code about how HDFS distributes the data. Al=
so how the map reduce engine tries to read the data. <br><br><br>Any hint re=
garding the location of those in the source code is appreciated.<br></span>&=
nbsp;<br></div><div><div><font style=3D"BACKGROUND-COLOR:#ffffff;" color=3D"=
#0080ff" face=3D"arial, helvetica, sans-serif" size=3D"2"><span style=3D"col=
or:rgb(0, 0, 0);">Regards,<br>Mahmood</span><b><br></b></font></div></div></=
div></div></blockquote></body></html>=

--Apple-Mail-1AA78B01-3657-4F6C-B704-5124FAD1D9D0--