hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Nauroth <cnaur...@hortonworks.com>
Subject Re: Maven build YARN ResourceManager only
Date Sat, 13 Apr 2013 04:41:26 GMT
I don't have an answer to your exact question, but I do have a different
suggestion that prevents the need to do frequent rebuilds of the whole
Hadoop source tree.  First, do a full build of the distribution tar.gz.
 Extract it and set up a custom hadoop-env.sh for yourself.  Inside the
hadoop-env.sh file, export the environment variables
you want to prepend before the classes loaded from the distribution.  For
example, this is what I have in mine right now, because I'm mostly working
on HDFS and NodeManager:


For your ResourceManager work, you could set up your HADOOP_CLASSPATH to
point at your hadoop-yarn-server-resourcemanager/target/classes directory.
 Then, source (.) this hadoop-env.sh in any shell that you're using to run
hadoop commands.  The daemons will print their full classpath before
launching, so you can check that to see if it worked.

With all of this in place, you can keep recompiling just
hadoop-yarn-server-resourcemanager whenever you make changes instead of the
whole hadoop-common tree.  Does this help?


On Fri, Apr 12, 2013 at 8:46 PM, Chin-Jung Hsu <oxhead.list@gmail.com>wrote:

> I am implementing my own YARN scheduler under 2.0.3-alpha.  Is that
> possible to build only the ResourceManager project, and then create a
> distribution tar.gz for the entire Hadoop project?  Right now, the
> compiling time takes me about 9 minutes.
> Thanks,
> oxhead

View raw message