hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karim Awara <karim.aw...@kaust.edu.sa>
Subject Re: modify HDFS
Date Thu, 03 Oct 2013 13:51:22 GMT
Hi all,

I checked out hadoop 2.1.0 beta  through this command  "svn checkout
http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta/hadoop-common-2.1.0-beta"
and i built all the subprojects using maven and imported them to eclipse.

Now if I modified the HDFS code, how shall i test the hdfs subproject?
(knowing that the project is missing the configuration files to actually be
deployed on the cluster).



--
Best Regards,
Karim Ahmed Awara


On Wed, Oct 2, 2013 at 10:31 PM, Ravi Prakash <ravihoo@ymail.com> wrote:

> Karim!
>
> Hadoop 3.0 corresponds to trunk currently. I would recommend you to use
> branch-2. Its fairly stable. hadoop-1.x is rather old and is in maintenance
> mode now. You can get all the branches from
> https://wiki.apache.org/hadoop/GitAndHadoop
>
> git clone git://git.apache.org/hadoop-common.git
>
> Please peruse https://hadoop.apache.org/docs/stable/cluster_setup.html to
> configure a distributed cluster. The block files should be in dfs.data.dir
> on the data nodes. You may also be able to use the Offline Image viewer to
> find all blocks and their locations from the fsimage file.
>
> HTH
> Ravi
>
>
>   ------------------------------
>  *From:* Pradeep Gollakota <pradeepg26@gmail.com>
> *To:* user@hadoop.apache.org
> *Sent:* Wednesday, October 2, 2013 9:29 AM
> *Subject:* Re: modify HDFS
>
> Since hadoop 3.0 is 2 major versions higher, it will be significantly
> different than working with hadoop 1.1.2. The hadoop-1.1 branch is
> available on SVN at
> http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.1/
>
>
>
>
> On Tue, Oct 1, 2013 at 11:30 PM, Karim Awara <karim.awara@kaust.edu.sa>wrote:
>
> Hi all,
>
> My previous web surfing led me to such steps that I executed successfully.
> However, my issue is that what version of hadoop this is? (I believe it is
> hadoop 3.0 since it supports maven build). However, I want to modify a
> stable release (hadoop 1.1.2, which does not have the maven build
> support).  Will working with the hadoop 3.0 make a lot of difference to me?
>
> Secondly, my goal is to modify the block placement strategy at HDFS in a
> distributed environment, and test such changes myself. Now, assuming I was
> successful in modifying the HDFS code, how to test such modifications
> (since the hadoop version is actually missing configuration files and so on
> that make it work in distributed environment)?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
>
> On Wed, Oct 2, 2013 at 1:13 AM, Ravi Prakash <ravihoo@ymail.com> wrote:
>
> Karim!
>
> You should read BUILDING.txt . I usually generate the eclipse files using
>
> mvn eclipse:eclipse
>
> Then I can import all the projects into eclipse as eclipse projects. This
> is useful for code navigation and completion etc. however I still build
> using command line:
> mvn -Pdist -Dmaven.javadoc.skip -DskipTests install
>
> HTH
> Ravi
>
>   ------------------------------
>  *From:* Jagat Singh <jagatsingh@gmail.com>
> *To:* user@hadoop.apache.org
> *Sent:* Tuesday, October 1, 2013 3:44 PM
> *Subject:* Re: modify HDFS
>
> Hi,
> What issue you having.
> I Wrote it about here , might help you
> Import it into eclipse as maven project
>
> http://jugnu-life.blogspot.com.au/2013/09/build-and-compile-hadoop-from-source.html?m=1
> Thanks
> On 01/10/2013 11:56 PM, "Karim Awara" <karim.awara@kaust.edu.sa> wrote:
>
> Hi,
>
> I want to modify the source code of HDFS to my usage, but I can't get any
> handy sources for development of hdfs on eclipse. (I tried the hdfs
> developer mailing list, but they are unresponsive). May you guide me?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
>
>
>
>
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
>
>
>
>
>

-- 

------------------------------
This message and its contents, including attachments are intended solely 
for the original recipient. If you are not the intended recipient or have 
received this message in error, please notify me immediately and delete 
this message from your computer system. Any unauthorized use or 
distribution is prohibited. Please consider the environment before printing 
this email.

Mime
View raw message