hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: Notes for working on mapreduce trunk after the MR-279 merge.
Date Thu, 18 Aug 2011 13:59:04 GMT
Awesome Vinod, thanks much for these instructions!

Perhaps these may be recorded on the frequented wiki HowToContribute
pages as well? :)

On Thu, Aug 18, 2011 at 6:19 PM, Vinod Kumar Vavilapalli
<vinodkv@hortonworks.com> wrote:
> MR-279 branch is merged into mapreduce trunk and this changes things a
> bit for developing on mapreduce.
>
> You can get all the help that is needed from the INSTALL file at
> http://svn.apache.org/repos/asf/hadoop/common/trunk/hadoop-mapreduce/INSTALL.
> Reproducing some of those contents here for the short-term lookup.
>
> ------------------------------------
> Checking out source code
> ------------------------------------
> svn checkout http://svn.apache.org/repos/asf/hadoop/common/trunk/hadoop-mapreduce
>
> --------------------------
> Directory structure
> --------------------------
>
> trunk/
>  - hadoop-mapreduce ( was mapreduce before)
>
> trunk/hadoop-mapreduce - Classic code. JT/TT reside here
>  - build.xml
>  - src
>
> trunk/hadoop-mapreduce/ - New code related to yarn reside here.
>  - assembly
>  - pom.xml
>  - hadoop-mr-client
>  - hadoop-yarn - Yarn APIs, libraries, and server code
>   -- hadoop-yarn-api
>   -- hadoop-yarn-common
>   -- hadoop-yarn-server - Server code, ResourceManager, NodeManager,
> server libraries and tests.
>      --- hadoop-yarn-server-common
>      --- hadoop-yarn-server-nodemanager
>      --- hadoop-yarn-server-resourcemanager
>      --- hadoop-yarn-server-tests
>  - hadoop-mr-client - MapReduce server and client code
>   -- hadoop-mapreduce-client-app
>   -- hadoop-mapreduce-client-core
>   -- hadoop-mapreduce-client-jobclient
>   -- hadoop-mapreduce-client-common
>   -- hadoop-mapreduce-client-hs
>   -- hadoop-mapreduce-client-shuffle
>
> -----------
> Building
> -----------
> Building yarn code and install into the local maven cache.
>  - mvn clean install
>  - In case you want to skip the tests run: mvn clean install -DskipTests
>
> Building classic code once yarn code is built.
>  - ant veryclean jar jar-test  -Dresolvers=internal
>
> ----------
> Eclipse
> -----------
>  1) For hacking on the new yarn+MR code in eclipse, you should run
> "mvn eclipse:eclipse" and then import the checked out source root as a
> maven project.
>  2) For developing on classic JT/TT code, running "ant eclipse" and
> importing as java project should continue to work.
>
> Hope that helps. If you run into issues, please send an email or
> create a JIRA issue.
>
> Thanks,
> +Vinod
>



-- 
Harsh J

Mime
View raw message