Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 36A7210D5F for ; Wed, 2 Oct 2013 19:32:13 +0000 (UTC) Received: (qmail 34226 invoked by uid 500); 2 Oct 2013 19:32:04 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 34090 invoked by uid 500); 2 Oct 2013 19:31:58 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 34079 invoked by uid 99); 2 Oct 2013 19:31:57 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 02 Oct 2013 19:31:57 +0000 X-ASF-Spam-Status: No, hits=4.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_BL_SPAMCOP_NET,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy includes SPF record at spf.trusted-forwarder.org) Received: from [216.109.114.159] (HELO nm41-vm4.bullet.mail.bf1.yahoo.com) (216.109.114.159) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 02 Oct 2013 19:31:49 +0000 Received: from [98.139.215.142] by nm41.bullet.mail.bf1.yahoo.com with NNFMP; 02 Oct 2013 19:31:27 -0000 Received: from [98.139.212.224] by tm13.bullet.mail.bf1.yahoo.com with NNFMP; 02 Oct 2013 19:31:27 -0000 Received: from [127.0.0.1] by omp1033.mail.bf1.yahoo.com with NNFMP; 02 Oct 2013 19:31:27 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 501036.81776.bm@omp1033.mail.bf1.yahoo.com Received: (qmail 41144 invoked by uid 60001); 2 Oct 2013 19:31:27 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ymail.com; s=s1024; t=1380742287; bh=iDf2GENOKrMDQnOYDpMjnLcZl1/auYG6h2vaCXmaVmw=; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type; b=nOdr/h/gsuDzeXv3iZpz/xFPXSCSApzjVZof8ztbsCwM3l2p1h0N++C6emKvV7kvVg32A1t8XAH+ddi+/6EOvL2eEdv0kYqbRhmGDgYPaFEBAO8vh0prwYqAUk9vXcNSpWj13Xrxp95KZLOJZePvGYmy/CbyF7/ImvpN18c7e1w= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=ymail.com; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type; b=qpnYNYmyp8r++y7jsaz8eRgfjiNsxXoLxF9mQn+zwecbeR4PJV4CFoGy438VqtNu+19XSUBL/XK058E1ObP2W37DWwFu4qb1ERqJ55vfmHUs5IvxeIoxbax43P8DBb8vIIUdnGZA+uRIg8rscJ4v1afIdvEACL3J4QVSTXXsmbE=; X-YMail-OSG: 5OmBXYEVM1kSNQjY.PilSJHArNQHzT1d9_HBQrkUyewb.C0 SeBZeu_A.D11AeTIuT4Ay1EIfdGyyLchS4TUyM.as_HUqU34KvsFgrokM482 ME1pSvm9BDxPR9SrM5iUcvxFmvJW7.NGBOld60dsIigZR7qSYqGZuXj6qACj ikNDht_Uim9I5w2wMhtWkUJn3uHtq_2o9_PM8pFcWZ7mlub6OzAkgN447eZQ S.88hgxz5laeiugFbXwpBPwIqGEBAJ11X8syZBVErkX8kJlWK8.JotAF2HPj 6zuD.w1qv61iDKoV2uB4gSSFQtZ165tRJJuDHqMEgAfIWTG_S4tc97ILhTZV DNuLKMgfAcfoOJ3t1Ti4WaN5uNLHu89UNXtPFQLefBuLBER4v.sNUB91Vmnw nGc.ILvosjO1UQ8fl_m3yx8XsPmGFm0451tx1UosfkNAY7EgHMtHe6IdK09p SZ.3Ef7qvqGgUxmswM6Od6TQRQwy5TPxlWBtiiHtsfppr6tL54vKN2gocB3N LPw2Ov5pXrVIa33J9mmWoVg1hPkNrX4Hp9rNsY_9qZlnQMJ_z430ZJMOb0gq Z1kY8NBWymxWsrs4LG6WebGR4eqeMNPqB1Xi2YmYf18JbKsoS9xDWH4D8PQI 3BElp7MMSVWDztTk_xiP1bye9wKVo9DrjdFF3Ap6vK8tCYPc76VDCfaG0BpR h0lDFTBY_xar8dLWVbwRC53GAScKe55v_LcVZmHWX9u567RjfP_JMq_aDyer mgngSYg-- Received: from [216.145.54.158] by web141203.mail.bf1.yahoo.com via HTTP; Wed, 02 Oct 2013 12:31:26 PDT X-Rocket-MIMEInfo: 002.001,S2FyaW0hCgpIYWRvb3AgMy4wIGNvcnJlc3BvbmRzIHRvIHRydW5rIGN1cnJlbnRseS4gSSB3b3VsZCByZWNvbW1lbmQgeW91IHRvIHVzZSBicmFuY2gtMi4gSXRzIGZhaXJseSBzdGFibGUuIGhhZG9vcC0xLnggaXMgcmF0aGVyIG9sZCBhbmQgaXMgaW4gbWFpbnRlbmFuY2UgbW9kZSBub3cuIFlvdSBjYW4gZ2V0IGFsbCB0aGUgYnJhbmNoZXMgZnJvbSBodHRwczovL3dpa2kuYXBhY2hlLm9yZy9oYWRvb3AvR2l0QW5kSGFkb29wCgpnaXQgY2xvbmUgZ2l0Oi8vZ2l0LmFwYWNoZS5vcmcvaGFkb29wLWNvbW1vbi4BMAEBAQE- X-Mailer: YahooMailWebService/0.8.160.587 References: <1380665631.64654.YahooMailNeo@web141204.mail.bf1.yahoo.com> Message-ID: <1380742286.34830.YahooMailNeo@web141203.mail.bf1.yahoo.com> Date: Wed, 2 Oct 2013 12:31:26 -0700 (PDT) From: Ravi Prakash Reply-To: Ravi Prakash Subject: Re: modify HDFS To: "user@hadoop.apache.org" In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="1123101620-682241773-1380742286=:34830" X-Virus-Checked: Checked by ClamAV on apache.org --1123101620-682241773-1380742286=:34830 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Karim!=0A=0AHadoop 3.0 corresponds to trunk currently. I would recommend yo= u to use branch-2. Its fairly stable. hadoop-1.x is rather old and is in ma= intenance mode now. You can get all the branches from https://wiki.apache.o= rg/hadoop/GitAndHadoop=0A=0Agit clone git://git.apache.org/hadoop-common.gi= t=0APlease perusehttps://hadoop.apache.org/docs/stable/cluster_setup.html t= o configure a distributed cluster. The block files should be in dfs.data.di= r on the data nodes. You may also be able to use the Offline Image viewer t= o find all blocks and their locations from the fsimage file.=0A=0AHTH=0ARav= i=0A=0A=0A=0A=0A________________________________=0A From: Pradeep Gollakota= =0ATo: user@hadoop.apache.org =0ASent: Wednesday, Oc= tober 2, 2013 9:29 AM=0ASubject: Re: modify HDFS=0A =0A=0A=0ASince hadoop 3= .0 is 2 major versions higher, it will be significantly different than work= ing with hadoop 1.1.2. The hadoop-1.1 branch is available on SVN at=A0http:= //svn.apache.org/repos/asf/hadoop/common/branches/branch-1.1/=0A=0A=0A=0A= =0A=0AOn Tue, Oct 1, 2013 at 11:30 PM, Karim Awara wrote:=0A=0AHi all, =0A>=0A>=0A>My previous web surfing led me to such s= teps that I executed successfully. However, my issue is that what version o= f hadoop this is? (I believe it is hadoop 3.0 since it supports maven build= ). However, I want to modify a stable release (hadoop 1.1.2, which does not= have the maven build support).=A0 Will working with the hadoop 3.0 make a = lot of difference to me? =0A>=0A>=0A>Secondly, my goal is to modify the blo= ck placement strategy at HDFS in a distributed environment, and test such c= hanges myself. Now, assuming I was successful in modifying the HDFS code, h= ow to test such modifications (since the hadoop version is actually missing= configuration files and so on that make it work in distributed environment= )? =0A>=0A>=0A>=0A>--=0A>Best Regards,=0A>Karim Ahmed Awara=0A>=0A>=0A>On W= ed, Oct 2, 2013 at 1:13 AM, Ravi Prakash wrote:=0A>=0A>= Karim!=0A>>=0A>>=0A>>You should read BUILDING.txt . I usually generate the = eclipse files using =0A>>=0A>>=0A>>=0A>>mvn eclipse:eclipse=0A>>=0A>>=0A>>T= hen I can import all the projects into eclipse as eclipse projects. This is= useful for code navigation and completion etc. however I still build using= command line:=0A>>mvn -Pdist -Dmaven.javadoc.skip -DskipTests install=0A>>= =0A>>HTH=0A>>Ravi=0A>>=0A>>=0A>>=0A>>=0A>>________________________________= =0A>> From: Jagat Singh =0A>>To: user@hadoop.apache.o= rg =0A>>Sent: Tuesday, October 1, 2013 3:44 PM=0A>>Subject: Re: modify HDFS= =0A>> =0A>>=0A>>=0A>>Hi,=0A>>What issue you having.=0A>>I Wrote it about he= re , might help you=0A>>Import it into eclipse as maven project=0A>>http://= jugnu-life.blogspot.com.au/2013/09/build-and-compile-hadoop-from-source.htm= l?m=3D1=0A>>Thanks=0A>>On 01/10/2013 11:56 PM, "Karim Awara" wrote:=0A>>=0A>>Hi, =0A>>>=0A>>>I want to modify the source co= de of HDFS to my usage, but I can't get any handy sources for development o= f hdfs on eclipse. (I tried the hdfs developer mailing list, but they are u= nresponsive). May you guide me?=0A>>>=0A>>>=0A>>>=0A>>>--=0A>>>Best Regards= ,=0A>>>Karim Ahmed Awara=0A>>>=0A>>>=0A>>>________________________________= =0A>>>This message and its contents, including attachments are intended sol= ely for the original recipient. If you are not the intended recipient or ha= ve received this message in error, please notify me immediately and delete = this message from your computer system. Any unauthorized use or distributio= n is prohibited. Please consider the environment before printing this email= .=0A>>=0A>>=0A>=0A>=0A>=0A>=0A>________________________________=0A>This mes= sage and its contents, including attachments are intended solely for the or= iginal recipient. If you are not the intended recipient or have received th= is message in error, please notify me immediately and delete this message f= rom your computer system. Any unauthorized use or distribution is prohibite= d. Please consider the environment before printing this email. --1123101620-682241773-1380742286=:34830 Content-Type: text/html; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable
Karim!

Hadoop = 3.0 corresponds to trunk currently. I would recommend you to use branch-2. = Its fairly stable. hadoop-1.x is rather old and is in maintenance mode now.= You can get all the branches from https://wiki.apache.org/hadoop/GitAndHad= oop
git clone git://git.apache.org/hadoop-common.git
Plea= se peruse https://hadoop.apache.org/docs/stable/cluster= _setup.html to configure a distributed cluster. The block files shou= ld be in dfs.data.dir on the data nodes. You may also be able to use the Of= fline Image viewer to find all blocks and their locations from the fsimage = file.

HTH
Ravi


From: Pradeep Gollakota <pradeepg26@= gmail.com>
To: user= @hadoop.apache.org
Sent:<= /b> Wednesday, October 2, 2013 9:29 AM
Subject: Re: modify HDFS

Since = hadoop 3.0 is 2 major versions higher, it will be significantly different t= han working with hadoop 1.1.2. The hadoop-1.1 branch is available on SVN at http://svn.apache.org/repos/asf= /hadoop/common/branches/branch-1.1/
=0A



On = Tue, Oct 1, 2013 at 11:30 PM, Karim Awara <karim.awara@kaust.edu.= sa> wrote:
=0A
Hi all,

My previous web surfing led me to such steps that = I executed successfully. However, my issue is that what version of hadoop t= his is? (I believe it is hadoop 3.0 since it supports maven build). However= , I want to modify a stable release (hadoop 1.1.2, which does not have the = maven build support).  Will working with the hadoop 3.0 make a lot of = difference to me?
=0A=0A=0A
Secondly, my goal is to modify the block placement strategy at HDFS in a d= istributed environment, and test such changes myself. Now, assuming I was s= uccessful in modifying the HDFS code, how to test such modifications (since= the hadoop version is actually missing configuration files and so on that = make it work in distributed environment)?
=0A=0A=0A

--
Best = Regards,
Karim Ahmed Awara
=0A

On Wed, Oct 2, 2013 at 1:13 AM, Ravi Prakash = <ravi= hoo@ymail.com> wrote:
=0A=0A=0A
Karim!

=0A=0A=0AYou should read BUILDING.txt . I usually generate the eclipse f= iles using

=0A=0A=0A
mvn eclipse:eclipse
=0A=0A=0A
Then I= can import all the projects into eclipse as eclipse projects. This is usef= ul for code navigation and completion etc. however I still build using comm= and line:
=0A=0A=0A
mvn -Pdist -Dmaven.javadoc.skip -DskipTests install

HTH
Ravi
<= div style=3D"font-style:normal;font-size:13.3333px;background-color:transpa= rent;font-family:times new roman, new york, times, serif;">=0A=0A=0A
From: J= agat Singh <jagat= singh@gmail.com>
=0A=0A=0A To: user@hadoop.apache.org
Sent: Tuesday, October 1, 2013 3:44 PM=0A=0A=0A Subject: Re: modify HDFS
Hi,
=0A
Wh= at issue you having.
=0A
I Wrote it about here , might= help you
=0A
Import it into eclipse as maven project<= /div>=0A=0A=0A=0A=0A
Thanks
=0A
On 01/10/2013 11:56 PM, "Karim Awara" <karim.awara@kau= st.edu.sa> wrote:
=0A=0A=0A=0A
Hi,

I want to mod= ify the source code of HDFS to my usage, but I can't get any handy sources = for development of hdfs on eclipse. (I tried the hdfs developer mailing lis= t, but they are unresponsive). May you guide me?
=0A=0A= =0A=0A=0A=0A=0A

--
Best Regards,
Karim Ahmed Awara
=0A=
=0A=0A
=0A
This message and its contents, including attachments are = intended solely for the original recipient. If you are not the intended rec= ipient or have received this message in error, please notify me immediately= and delete this message from your computer system. Any unauthorized use or= distribution is prohibited. Please consider the environment before printin= g this email.
=0A=0A=0A=0A



=0A=0A
= =0A
This message and its cont= ents, including attachments are intended solely for the original recipient.= If you are not the intended recipient or have received this message in err= or, please notify me immediately and delete this message from your computer= system. Any unauthorized use or distribution is prohibited. Please conside= r the environment before printing this email.
=0A



=
--1123101620-682241773-1380742286=:34830--