hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeffrey Naisbitt <jnais...@yahoo-inc.com>
Subject Re: getting started building Mavenized hadoop common
Date Tue, 02 Aug 2011 21:55:59 GMT
Thanks for all your work and updates on this, Alejandro!  It's much
better/easier to work with :)

I did have a few issues/questions:
First, I still had to manually add 'target/generated-src/test/java' to my
build path sources in eclipse.  I don't know if this is due to something I
did wrong, but I would think this should be automatic.

Also, I ran into a few issues with the test-patch.sh script:
First, it will fail if MAVEN_HOME is not set, and I didn't see anything
about that in documentation.
Also, it gives a couple of non-critical errors:
./dev-support/test-patch.sh: line 578: auxwww: command not found
./dev-support/test-patch.sh: line 578: /usr/bin/nawk: No such file or
directory
The first is because $PS is not set (and was previously passed in for the
HUDSON version), and the second is just because my box doesn┬╣t have nawk on
it.

Thanks again.
-Jeff


On 8/2/11 4:13 PM, "Alejandro Abdelnur" <tucu@cloudera.com> wrote:

> With the HADOOP-6671 commit the way of building hadoop common has changed
> significantly.
> 
> While the wiki explains these changes, and there is a BUILDING.txt
> directory, still I guess things will hit many of of you.
> 
> Because of this I've put together some brief notes.
> 
> Thanks.
> 
> Alejandro
> 
> ----------------------
> NEW LAYOUT
> 
> After updating the trunk you'll the the following directory changes at top
> level
> 
>  Removed: common/
>  New: hadoop-common/, hadoop-project/, hadoop-annotations/,
> hadoop-assemblies/
> 
> * hadoop-common/ is the new common/ and its sub-dirs are organized following
> Maven standard project layout.
> * hadoop-project/ contains Hadoop project root POM, all dependency versions
> are defined there
> * hadoop-annotations/ contains the Hadoop public/private annotation classes
> * hadoop-assemblies/ contains the assembly files that create the
> distribution directories layout
> 
> ----------------------
> BUILDING REQUIREMENTS
> 
> The only new build requirement is Maven 3 (it must be at least Maven 3).
> 
> The environment var FORREST_HOME must be set if building the documentation.
> 
> ----------------------
> FIRST MAVEN BUILD
> 
> It must be run from the trunk/ directory.
> 
> Run: 'mvn install -DskipTests'
> 
> This will install the different submodules
> (project/annotations/assemblies/common) into the local Maven cache
> (~/.m2/repository).
> 
> After this is done, you can build from the hadoop-commons directory.
> 
> NOTE: this will not be required once the SNAPSHOTS Maven repo has the
> snapshots published.
> ----------------------
> TARGET/ IS THE NEW BUILD/
> 
> The new build directory is target/
> 
> ----------------------
> USING AN IDE
> 
> Eclipse and IntelliJ will recognize and open the project from the POM file.
> 
> Make sure you run a 'mvn test -DskipTests' every time you have a clean
> target/ directory as Maven generates code required for testing and sets some
> directories under target/
> 
> ----------------------
> BUILDING
> 
> Run 'mvn compile'
> 
> To compile native code add '-Pnative'
> 
> ----------------------
> RUNNING TESTCASES
> 
> Run 'mvn test -Dtest=TESTCASECLASS'
> 
> To run multiple testcases separate the testcases name with comma
> 
> To run all testcases don't specifiy -'Dtest=...'
> 
> NOTE: TESTCASECLASS is just the testcase classname, no package name, no
> extension.
> 
> ----------------------
> CREATING THE TAR
> 
> Run 'mvn package -Pbintar -DskipTests'
> 
> NOTE: The '-Ptar' profile will create the legacy layout, but the Hadoop
> scripts will not work with the legacy layout (this has been the case before
>  HADOOP-6671)
> 
> ----------------------
> RUNNING THE HADOOP SCRIPTS IN DEVELOPMENT
> 
> Run 'mvn package -Pbintar -DskipTests'
> 
> The Hadoop scripts can be executed from
> hadoop-common/target/hadoop-common-0.23.0-SNAPSHOT-bin/bin/ directory.
> 
> ----------------------


Mime
View raw message