hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Graves <tgra...@yahoo-inc.com>
Subject Re: Notes for working on mapreduce trunk after the MR-279 merge.
Date Thu, 18 Aug 2011 15:40:58 GMT

I just built and brought up mrv2 on a single node cluster.  I was
successfully able to run wordcount from the mapreduce-examples after adding
the following config change to my mapred-site.xml.  Without this change it
was still trying to contact the jobtracker.

<property>
<name> mapreduce.framework.name</name>
<value>yarn</value>
</property>

I'll file a jira and update the INSTALL doc.

Tom

On 8/18/11 7:49 AM, "Vinod Kumar Vavilapalli" <vinodkv@hortonworks.com>
wrote:

> MR-279 branch is merged into mapreduce trunk and this changes things a
> bit for developing on mapreduce.
> 
> You can get all the help that is needed from the INSTALL file at
> http://svn.apache.org/repos/asf/hadoop/common/trunk/hadoop-mapreduce/INSTALL.
> Reproducing some of those contents here for the short-term lookup.
> 
> ------------------------------------
> Checking out source code
> ------------------------------------
> svn checkout 
> http://svn.apache.org/repos/asf/hadoop/common/trunk/hadoop-mapreduce
> 
> --------------------------
> Directory structure
> --------------------------
> 
> trunk/
>   - hadoop-mapreduce ( was mapreduce before)
> 
> trunk/hadoop-mapreduce - Classic code. JT/TT reside here
>  - build.xml
>  - src
> 
> trunk/hadoop-mapreduce/ - New code related to yarn reside here.
>  - assembly
>  - pom.xml
>  - hadoop-mr-client
>  - hadoop-yarn - Yarn APIs, libraries, and server code
>    -- hadoop-yarn-api
>    -- hadoop-yarn-common
>    -- hadoop-yarn-server - Server code, ResourceManager, NodeManager,
> server libraries and tests.
>       --- hadoop-yarn-server-common
>       --- hadoop-yarn-server-nodemanager
>       --- hadoop-yarn-server-resourcemanager
>       --- hadoop-yarn-server-tests
>  - hadoop-mr-client - MapReduce server and client code
>    -- hadoop-mapreduce-client-app
>    -- hadoop-mapreduce-client-core
>    -- hadoop-mapreduce-client-jobclient
>    -- hadoop-mapreduce-client-common
>    -- hadoop-mapreduce-client-hs
>    -- hadoop-mapreduce-client-shuffle
> 
> -----------
> Building
> -----------
> Building yarn code and install into the local maven cache.
>  - mvn clean install
>  - In case you want to skip the tests run: mvn clean install -DskipTests
> 
> Building classic code once yarn code is built.
>  - ant veryclean jar jar-test  -Dresolvers=internal
> 
> ----------
> Eclipse
> -----------
>  1) For hacking on the new yarn+MR code in eclipse, you should run
> "mvn eclipse:eclipse" and then import the checked out source root as a
> maven project.
>  2) For developing on classic JT/TT code, running "ant eclipse" and
> importing as java project should continue to work.
> 
> Hope that helps. If you run into issues, please send an email or
> create a JIRA issue.
> 
> Thanks,
> +Vinod


Mime
View raw message