hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Glen Mazza <gma...@talend.com>
Subject Re: development environment for hadoop core
Date Wed, 16 Jan 2013 13:31:24 GMT
On 01/15/2013 06:50 PM, Erik Paulson wrote:
> Hello -
> I'm curious what Hadoop developers use for their day-to-day hacking on
> Hadoop. I'm talking changes to the Hadoop libraries and daemons, and not
> developing Map-Reduce jobs or using using the HDFS Client libraries to talk
> to a filesystem from an application.
> I've checked out Hadoop, made minor changes and built it with Maven, and
> tracked down the resulting artifacts in a target/ directory that I could
> deploy. Is this typically how a cloudera/hortonworks/mapr/etc dev works, or
> are the IDEs more common?
I haven't built Hadoop yet myself.  Your use of "a" in "a target/ 
directory" indicates you're also kind of new with Maven itself, as 
that's the standard output folder for any Maven project.  One of many 
nice things about Maven is once you learn how to build one project with 
it you pretty much know how to build any project with it, as 
everything's standardized with it.

Probably best to stick with the command line for building and use 
Eclipse for editing, to keep things simple, but don't forget the mvn 
eclipse:eclipse command to set up Eclipse projects that you can 
subsequently import into your Eclipse IDE: 

> I realize this sort of sounds like a dumb question, but I'm mostly curious
> what I might be missing out on if I stay away from anything other than vim,
> and not being entirely sure where maven might be caching jars that it uses
> to build,

That will be your local Maven repository, in an .m2 hidden folder in 
your user home directory.

> and how careful I have to be to ensure that my changes wind up in
> the right places without having to do a clean build every time.

Maven can detect changes (using mvn install instead of mvn clean 
install), but I prefer doing clean builds.  You can use the 
-Dmaven.test.skip setting to speed up your "mvn clean installs" if you 
don't wish to run the tests each time.


> Thanks!
> -Erik

Glen Mazza
Talend Community Coders - coders.talend.com
blog: www.jroller.com/gmazza

View raw message