hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alejandro Abdelnur <t...@cloudera.com>
Subject Re: getting started building Mavenized hadoop common
Date Tue, 02 Aug 2011 22:21:09 GMT
Jeffrey,

Thanks.

Regarding adding the 'target/generated-src/test/java' dir to the build path.
You are correct, you have to add it manually to your IDE (I use IntelliJ and
it is the same story). But unless you need to debug through the generated
code you don't need to do so (doing a 'mvn test -DskipTests' will
generate/compile the class and the .class file will be in the IDE project
classpath).

Regarding MAVEN_HOME, I don't have it in my environment and the build works.

Regarding the test-patch.sh issues, was it working for you prior to
HADOOP-6671? Because we didn't change that line. Also, keep in mind that the
injection fault tests are not wired yet.

Thanks.

Alejandro

On Tue, Aug 2, 2011 at 2:55 PM, Jeffrey Naisbitt <jnaisbit@yahoo-inc.com>wrote:

> Thanks for all your work and updates on this, Alejandro!  It's much
> better/easier to work with :)
>
> I did have a few issues/questions:
> First, I still had to manually add 'target/generated-src/test/java' to my
> build path sources in eclipse.  I don't know if this is due to something I
> did wrong, but I would think this should be automatic.
>
> Also, I ran into a few issues with the test-patch.sh script:
> First, it will fail if MAVEN_HOME is not set, and I didn't see anything
> about that in documentation.
> Also, it gives a couple of non-critical errors:
> ./dev-support/test-patch.sh: line 578: auxwww: command not found
> ./dev-support/test-patch.sh: line 578: /usr/bin/nawk: No such file or
> directory
> The first is because $PS is not set (and was previously passed in for the
> HUDSON version), and the second is just because my box doesn┬╣t have nawk on
> it.
>
> Thanks again.
> -Jeff
>
>
> On 8/2/11 4:13 PM, "Alejandro Abdelnur" <tucu@cloudera.com> wrote:
>
> > With the HADOOP-6671 commit the way of building hadoop common has changed
> > significantly.
> >
> > While the wiki explains these changes, and there is a BUILDING.txt
> > directory, still I guess things will hit many of of you.
> >
> > Because of this I've put together some brief notes.
> >
> > Thanks.
> >
> > Alejandro
> >
> > ----------------------
> > NEW LAYOUT
> >
> > After updating the trunk you'll the the following directory changes at
> top
> > level
> >
> >  Removed: common/
> >  New: hadoop-common/, hadoop-project/, hadoop-annotations/,
> > hadoop-assemblies/
> >
> > * hadoop-common/ is the new common/ and its sub-dirs are organized
> following
> > Maven standard project layout.
> > * hadoop-project/ contains Hadoop project root POM, all dependency
> versions
> > are defined there
> > * hadoop-annotations/ contains the Hadoop public/private annotation
> classes
> > * hadoop-assemblies/ contains the assembly files that create the
> > distribution directories layout
> >
> > ----------------------
> > BUILDING REQUIREMENTS
> >
> > The only new build requirement is Maven 3 (it must be at least Maven 3).
> >
> > The environment var FORREST_HOME must be set if building the
> documentation.
> >
> > ----------------------
> > FIRST MAVEN BUILD
> >
> > It must be run from the trunk/ directory.
> >
> > Run: 'mvn install -DskipTests'
> >
> > This will install the different submodules
> > (project/annotations/assemblies/common) into the local Maven cache
> > (~/.m2/repository).
> >
> > After this is done, you can build from the hadoop-commons directory.
> >
> > NOTE: this will not be required once the SNAPSHOTS Maven repo has the
> > snapshots published.
> > ----------------------
> > TARGET/ IS THE NEW BUILD/
> >
> > The new build directory is target/
> >
> > ----------------------
> > USING AN IDE
> >
> > Eclipse and IntelliJ will recognize and open the project from the POM
> file.
> >
> > Make sure you run a 'mvn test -DskipTests' every time you have a clean
> > target/ directory as Maven generates code required for testing and sets
> some
> > directories under target/
> >
> > ----------------------
> > BUILDING
> >
> > Run 'mvn compile'
> >
> > To compile native code add '-Pnative'
> >
> > ----------------------
> > RUNNING TESTCASES
> >
> > Run 'mvn test -Dtest=TESTCASECLASS'
> >
> > To run multiple testcases separate the testcases name with comma
> >
> > To run all testcases don't specifiy -'Dtest=...'
> >
> > NOTE: TESTCASECLASS is just the testcase classname, no package name, no
> > extension.
> >
> > ----------------------
> > CREATING THE TAR
> >
> > Run 'mvn package -Pbintar -DskipTests'
> >
> > NOTE: The '-Ptar' profile will create the legacy layout, but the Hadoop
> > scripts will not work with the legacy layout (this has been the case
> before
> >  HADOOP-6671)
> >
> > ----------------------
> > RUNNING THE HADOOP SCRIPTS IN DEVELOPMENT
> >
> > Run 'mvn package -Pbintar -DskipTests'
> >
> > The Hadoop scripts can be executed from
> > hadoop-common/target/hadoop-common-0.23.0-SNAPSHOT-bin/bin/ directory.
> >
> > ----------------------
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message