hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeffrey Naisbitt <jnais...@yahoo-inc.com>
Subject Re: getting started building Mavenized hadoop common
Date Tue, 02 Aug 2011 22:47:20 GMT
On 8/2/11 5:21 PM, "Alejandro Abdelnur" <tucu@cloudera.com> wrote:
> Regarding adding the 'target/generated-src/test/java' dir to the build path.
> You are correct, you have to add it manually to your IDE (I use IntelliJ and
> it is the same story). But unless you need to debug through the generated
> code you don't need to do so (doing a 'mvn test -DskipTests' will
> generate/compile the class and the .class file will be in the IDE project
> classpath).

I like to debug through the code :)  It would be nice if there were an
automated way to handle that folder, but in the meantime, it would probably
be useful to document that along with the eclipse instructions.


> Regarding MAVEN_HOME, I don't have it in my environment and the build works.
I was referring to running test-patch.sh.  (test-patch.sh now requires the
MAVEN_HOME setting).  I'm ok if this is a requirement to run it now (like
ANT_HOME was required before), but it should probably be mentioned somewhere
since I didn't think about it :)

> Regarding the test-patch.sh issues, was it working for you prior to
> HADOOP-6671? Because we didn't change that line. Also, keep in mind that the
> injection fault tests are not wired yet.

I don't remember seeing those before, but yeah, it doesn't look like your
patch touched anything to do with them.  They seem to be ignored anyway.

Thanks again...great work :)
-Jeff




> On Tue, Aug 2, 2011 at 2:55 PM, Jeffrey Naisbitt
> <jnaisbit@yahoo-inc.com>wrote:
> 
>> Thanks for all your work and updates on this, Alejandro!  It's much
>> better/easier to work with :)
>> 
>> I did have a few issues/questions:
>> First, I still had to manually add 'target/generated-src/test/java' to my
>> build path sources in eclipse.  I don't know if this is due to something I
>> did wrong, but I would think this should be automatic.
>> 
>> Also, I ran into a few issues with the test-patch.sh script:
>> First, it will fail if MAVEN_HOME is not set, and I didn't see anything
>> about that in documentation.
>> Also, it gives a couple of non-critical errors:
>> ./dev-support/test-patch.sh: line 578: auxwww: command not found
>> ./dev-support/test-patch.sh: line 578: /usr/bin/nawk: No such file or
>> directory
>> The first is because $PS is not set (and was previously passed in for the
>> HUDSON version), and the second is just because my box doesn┬╣t have nawk on
>> it.
>> 
>> Thanks again.
>> -Jeff
>> 
>> 
>> On 8/2/11 4:13 PM, "Alejandro Abdelnur" <tucu@cloudera.com> wrote:
>> 
>>> With the HADOOP-6671 commit the way of building hadoop common has changed
>>> significantly.
>>> 
>>> While the wiki explains these changes, and there is a BUILDING.txt
>>> directory, still I guess things will hit many of of you.
>>> 
>>> Because of this I've put together some brief notes.
>>> 
>>> Thanks.
>>> 
>>> Alejandro
>>> 
>>> ----------------------
>>> NEW LAYOUT
>>> 
>>> After updating the trunk you'll the the following directory changes at
>> top
>>> level
>>> 
>>>  Removed: common/
>>>  New: hadoop-common/, hadoop-project/, hadoop-annotations/,
>>> hadoop-assemblies/
>>> 
>>> * hadoop-common/ is the new common/ and its sub-dirs are organized
>> following
>>> Maven standard project layout.
>>> * hadoop-project/ contains Hadoop project root POM, all dependency
>> versions
>>> are defined there
>>> * hadoop-annotations/ contains the Hadoop public/private annotation
>> classes
>>> * hadoop-assemblies/ contains the assembly files that create the
>>> distribution directories layout
>>> 
>>> ----------------------
>>> BUILDING REQUIREMENTS
>>> 
>>> The only new build requirement is Maven 3 (it must be at least Maven 3).
>>> 
>>> The environment var FORREST_HOME must be set if building the
>> documentation.
>>> 
>>> ----------------------
>>> FIRST MAVEN BUILD
>>> 
>>> It must be run from the trunk/ directory.
>>> 
>>> Run: 'mvn install -DskipTests'
>>> 
>>> This will install the different submodules
>>> (project/annotations/assemblies/common) into the local Maven cache
>>> (~/.m2/repository).
>>> 
>>> After this is done, you can build from the hadoop-commons directory.
>>> 
>>> NOTE: this will not be required once the SNAPSHOTS Maven repo has the
>>> snapshots published.
>>> ----------------------
>>> TARGET/ IS THE NEW BUILD/
>>> 
>>> The new build directory is target/
>>> 
>>> ----------------------
>>> USING AN IDE
>>> 
>>> Eclipse and IntelliJ will recognize and open the project from the POM
>> file.
>>> 
>>> Make sure you run a 'mvn test -DskipTests' every time you have a clean
>>> target/ directory as Maven generates code required for testing and sets
>> some
>>> directories under target/
>>> 
>>> ----------------------
>>> BUILDING
>>> 
>>> Run 'mvn compile'
>>> 
>>> To compile native code add '-Pnative'
>>> 
>>> ----------------------
>>> RUNNING TESTCASES
>>> 
>>> Run 'mvn test -Dtest=TESTCASECLASS'
>>> 
>>> To run multiple testcases separate the testcases name with comma
>>> 
>>> To run all testcases don't specifiy -'Dtest=...'
>>> 
>>> NOTE: TESTCASECLASS is just the testcase classname, no package name, no
>>> extension.
>>> 
>>> ----------------------
>>> CREATING THE TAR
>>> 
>>> Run 'mvn package -Pbintar -DskipTests'
>>> 
>>> NOTE: The '-Ptar' profile will create the legacy layout, but the Hadoop
>>> scripts will not work with the legacy layout (this has been the case
>> before
>>>  HADOOP-6671)
>>> 
>>> ----------------------
>>> RUNNING THE HADOOP SCRIPTS IN DEVELOPMENT
>>> 
>>> Run 'mvn package -Pbintar -DskipTests'
>>> 
>>> The Hadoop scripts can be executed from
>>> hadoop-common/target/hadoop-common-0.23.0-SNAPSHOT-bin/bin/ directory.
>>> 
>>> ----------------------
>> 
>> 


Mime
View raw message