hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ravi Prakash <ravi...@ymail.com>
Subject Re: modify HDFS
Date Wed, 02 Oct 2013 19:31:26 GMT
Karim!

Hadoop 3.0 corresponds to trunk currently. I would recommend you to use branch-2. Its fairly
stable. hadoop-1.x is rather old and is in maintenance mode now. You can get all the branches
from https://wiki.apache.org/hadoop/GitAndHadoop

git clone git://git.apache.org/hadoop-common.git
Please perusehttps://hadoop.apache.org/docs/stable/cluster_setup.html to configure a distributed
cluster. The block files should be in dfs.data.dir on the data nodes. You may also be able
to use the Offline Image viewer to find all blocks and their locations from the fsimage file.

HTH
Ravi




________________________________
 From: Pradeep Gollakota <pradeepg26@gmail.com>
To: user@hadoop.apache.org 
Sent: Wednesday, October 2, 2013 9:29 AM
Subject: Re: modify HDFS
 


Since hadoop 3.0 is 2 major versions higher, it will be significantly different than working
with hadoop 1.1.2. The hadoop-1.1 branch is available on SVN at http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.1/





On Tue, Oct 1, 2013 at 11:30 PM, Karim Awara <karim.awara@kaust.edu.sa> wrote:

Hi all, 
>
>
>My previous web surfing led me to such steps that I executed successfully. However, my
issue is that what version of hadoop this is? (I believe it is hadoop 3.0 since it supports
maven build). However, I want to modify a stable release (hadoop 1.1.2, which does not have
the maven build support).  Will working with the hadoop 3.0 make a lot of difference to me?

>
>
>Secondly, my goal is to modify the block placement strategy at HDFS in a distributed environment,
and test such changes myself. Now, assuming I was successful in modifying the HDFS code, how
to test such modifications (since the hadoop version is actually missing configuration files
and so on that make it work in distributed environment)? 
>
>
>
>--
>Best Regards,
>Karim Ahmed Awara
>
>
>On Wed, Oct 2, 2013 at 1:13 AM, Ravi Prakash <ravihoo@ymail.com> wrote:
>
>Karim!
>>
>>
>>You should read BUILDING.txt . I usually generate the eclipse files using 
>>
>>
>>
>>mvn eclipse:eclipse
>>
>>
>>Then I can import all the projects into eclipse as eclipse projects. This is useful
for code navigation and completion etc. however I still build using command line:
>>mvn -Pdist -Dmaven.javadoc.skip -DskipTests install
>>
>>HTH
>>Ravi
>>
>>
>>
>>
>>________________________________
>> From: Jagat Singh <jagatsingh@gmail.com>
>>To: user@hadoop.apache.org 
>>Sent: Tuesday, October 1, 2013 3:44 PM
>>Subject: Re: modify HDFS
>> 
>>
>>
>>Hi,
>>What issue you having.
>>I Wrote it about here , might help you
>>Import it into eclipse as maven project
>>http://jugnu-life.blogspot.com.au/2013/09/build-and-compile-hadoop-from-source.html?m=1
>>Thanks
>>On 01/10/2013 11:56 PM, "Karim Awara" <karim.awara@kaust.edu.sa> wrote:
>>
>>Hi, 
>>>
>>>I want to modify the source code of HDFS to my usage, but I can't get any handy
sources for development of hdfs on eclipse. (I tried the hdfs developer mailing list, but
they are unresponsive). May you guide me?
>>>
>>>
>>>
>>>--
>>>Best Regards,
>>>Karim Ahmed Awara
>>>
>>>
>>>________________________________
>>>This message and its contents, including attachments are intended solely for the
original recipient. If you are not the intended recipient or have received this message in
error, please notify me immediately and delete this message from your computer system. Any
unauthorized use or distribution is prohibited. Please consider the environment before printing
this email.
>>
>>
>
>
>
>
>________________________________
>This message and its contents, including attachments are intended solely for the original
recipient. If you are not the intended recipient or have received this message in error, please
notify me immediately and delete this message from your computer system. Any unauthorized
use or distribution is prohibited. Please consider the environment before printing this email.
Mime
View raw message