hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinod Kumar Vavilapalli <vino...@hortonworks.com>
Subject Re: Notes for working on mapreduce trunk after the MR-279 merge.
Date Thu, 18 Aug 2011 15:46:41 GMT
Good idea. Created http://wiki.apache.org/hadoop/DevelopingOnTrunkAfter279Merge

Please feel free to edit it. Once it stabilises, we can move the
content over to HowToContribute and/or something like
HowToBuildAndInstall.

Thanks,
+Vinod

On Thu, Aug 18, 2011 at 7:29 PM, Harsh J <harsh@cloudera.com> wrote:
> Awesome Vinod, thanks much for these instructions!
>
> Perhaps these may be recorded on the frequented wiki HowToContribute
> pages as well? :)
>
> On Thu, Aug 18, 2011 at 6:19 PM, Vinod Kumar Vavilapalli
> <vinodkv@hortonworks.com> wrote:
>> MR-279 branch is merged into mapreduce trunk and this changes things a
>> bit for developing on mapreduce.
>>
>> You can get all the help that is needed from the INSTALL file at
>> http://svn.apache.org/repos/asf/hadoop/common/trunk/hadoop-mapreduce/INSTALL.
>> Reproducing some of those contents here for the short-term lookup.
>>
>> ------------------------------------
>> Checking out source code
>> ------------------------------------
>> svn checkout http://svn.apache.org/repos/asf/hadoop/common/trunk/hadoop-mapreduce
>>
>> --------------------------
>> Directory structure
>> --------------------------
>>
>> trunk/
>>  - hadoop-mapreduce ( was mapreduce before)
>>
>> trunk/hadoop-mapreduce - Classic code. JT/TT reside here
>>  - build.xml
>>  - src
>>
>> trunk/hadoop-mapreduce/ - New code related to yarn reside here.
>>  - assembly
>>  - pom.xml
>>  - hadoop-mr-client
>>  - hadoop-yarn - Yarn APIs, libraries, and server code
>>   -- hadoop-yarn-api
>>   -- hadoop-yarn-common
>>   -- hadoop-yarn-server - Server code, ResourceManager, NodeManager,
>> server libraries and tests.
>>      --- hadoop-yarn-server-common
>>      --- hadoop-yarn-server-nodemanager
>>      --- hadoop-yarn-server-resourcemanager
>>      --- hadoop-yarn-server-tests
>>  - hadoop-mr-client - MapReduce server and client code
>>   -- hadoop-mapreduce-client-app
>>   -- hadoop-mapreduce-client-core
>>   -- hadoop-mapreduce-client-jobclient
>>   -- hadoop-mapreduce-client-common
>>   -- hadoop-mapreduce-client-hs
>>   -- hadoop-mapreduce-client-shuffle
>>
>> -----------
>> Building
>> -----------
>> Building yarn code and install into the local maven cache.
>>  - mvn clean install
>>  - In case you want to skip the tests run: mvn clean install -DskipTests
>>
>> Building classic code once yarn code is built.
>>  - ant veryclean jar jar-test  -Dresolvers=internal
>>
>> ----------
>> Eclipse
>> -----------
>>  1) For hacking on the new yarn+MR code in eclipse, you should run
>> "mvn eclipse:eclipse" and then import the checked out source root as a
>> maven project.
>>  2) For developing on classic JT/TT code, running "ant eclipse" and
>> importing as java project should continue to work.
>>
>> Hope that helps. If you run into issues, please send an email or
>> create a JIRA issue.
>>
>> Thanks,
>> +Vinod
>>
>
>
>
> --
> Harsh J
>

Mime
View raw message