hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tom White <...@cloudera.com>
Subject Re: getting started building Mavenized hadoop common
Date Wed, 03 Aug 2011 00:44:45 GMT
On Tue, Aug 2, 2011 at 3:47 PM, Jeffrey Naisbitt <jnaisbit@yahoo-inc.com> wrote:
> On 8/2/11 5:21 PM, "Alejandro Abdelnur" <tucu@cloudera.com> wrote:
>> Regarding adding the 'target/generated-src/test/java' dir to the build path.
>> You are correct, you have to add it manually to your IDE (I use IntelliJ and
>> it is the same story). But unless you need to debug through the generated
>> code you don't need to do so (doing a 'mvn test -DskipTests' will
>> generate/compile the class and the .class file will be in the IDE project
>> classpath).
>
> I like to debug through the code :)  It would be nice if there were an
> automated way to handle that folder, but in the meantime, it would probably
> be useful to document that along with the eclipse instructions.

I had to do this step too. I've added it to the instructions on
http://wiki.apache.org/hadoop/EclipseEnvironment, but I agree it would
be nice to automate this if anyone knows the relevant setting.

>
>
>> Regarding MAVEN_HOME, I don't have it in my environment and the build works.
> I was referring to running test-patch.sh.  (test-patch.sh now requires the
> MAVEN_HOME setting).  I'm ok if this is a requirement to run it now (like
> ANT_HOME was required before), but it should probably be mentioned somewhere
> since I didn't think about it :)

I've updated http://wiki.apache.org/hadoop/HowToContribute to mention
MAVEN_HOME.

Thanks,
Tom

>
>> Regarding the test-patch.sh issues, was it working for you prior to
>> HADOOP-6671? Because we didn't change that line. Also, keep in mind that the
>> injection fault tests are not wired yet.
>
> I don't remember seeing those before, but yeah, it doesn't look like your
> patch touched anything to do with them.  They seem to be ignored anyway.
>
> Thanks again...great work :)
> -Jeff
>
>
>
>
>> On Tue, Aug 2, 2011 at 2:55 PM, Jeffrey Naisbitt
>> <jnaisbit@yahoo-inc.com>wrote:
>>
>>> Thanks for all your work and updates on this, Alejandro!  It's much
>>> better/easier to work with :)
>>>
>>> I did have a few issues/questions:
>>> First, I still had to manually add 'target/generated-src/test/java' to my
>>> build path sources in eclipse.  I don't know if this is due to something I
>>> did wrong, but I would think this should be automatic.
>>>
>>> Also, I ran into a few issues with the test-patch.sh script:
>>> First, it will fail if MAVEN_HOME is not set, and I didn't see anything
>>> about that in documentation.
>>> Also, it gives a couple of non-critical errors:
>>> ./dev-support/test-patch.sh: line 578: auxwww: command not found
>>> ./dev-support/test-patch.sh: line 578: /usr/bin/nawk: No such file or
>>> directory
>>> The first is because $PS is not set (and was previously passed in for the
>>> HUDSON version), and the second is just because my box doesn¹t have nawk on
>>> it.
>>>
>>> Thanks again.
>>> -Jeff
>>>
>>>
>>> On 8/2/11 4:13 PM, "Alejandro Abdelnur" <tucu@cloudera.com> wrote:
>>>
>>>> With the HADOOP-6671 commit the way of building hadoop common has changed
>>>> significantly.
>>>>
>>>> While the wiki explains these changes, and there is a BUILDING.txt
>>>> directory, still I guess things will hit many of of you.
>>>>
>>>> Because of this I've put together some brief notes.
>>>>
>>>> Thanks.
>>>>
>>>> Alejandro
>>>>
>>>> ----------------------
>>>> NEW LAYOUT
>>>>
>>>> After updating the trunk you'll the the following directory changes at
>>> top
>>>> level
>>>>
>>>>  Removed: common/
>>>>  New: hadoop-common/, hadoop-project/, hadoop-annotations/,
>>>> hadoop-assemblies/
>>>>
>>>> * hadoop-common/ is the new common/ and its sub-dirs are organized
>>> following
>>>> Maven standard project layout.
>>>> * hadoop-project/ contains Hadoop project root POM, all dependency
>>> versions
>>>> are defined there
>>>> * hadoop-annotations/ contains the Hadoop public/private annotation
>>> classes
>>>> * hadoop-assemblies/ contains the assembly files that create the
>>>> distribution directories layout
>>>>
>>>> ----------------------
>>>> BUILDING REQUIREMENTS
>>>>
>>>> The only new build requirement is Maven 3 (it must be at least Maven 3).
>>>>
>>>> The environment var FORREST_HOME must be set if building the
>>> documentation.
>>>>
>>>> ----------------------
>>>> FIRST MAVEN BUILD
>>>>
>>>> It must be run from the trunk/ directory.
>>>>
>>>> Run: 'mvn install -DskipTests'
>>>>
>>>> This will install the different submodules
>>>> (project/annotations/assemblies/common) into the local Maven cache
>>>> (~/.m2/repository).
>>>>
>>>> After this is done, you can build from the hadoop-commons directory.
>>>>
>>>> NOTE: this will not be required once the SNAPSHOTS Maven repo has the
>>>> snapshots published.
>>>> ----------------------
>>>> TARGET/ IS THE NEW BUILD/
>>>>
>>>> The new build directory is target/
>>>>
>>>> ----------------------
>>>> USING AN IDE
>>>>
>>>> Eclipse and IntelliJ will recognize and open the project from the POM
>>> file.
>>>>
>>>> Make sure you run a 'mvn test -DskipTests' every time you have a clean
>>>> target/ directory as Maven generates code required for testing and sets
>>> some
>>>> directories under target/
>>>>
>>>> ----------------------
>>>> BUILDING
>>>>
>>>> Run 'mvn compile'
>>>>
>>>> To compile native code add '-Pnative'
>>>>
>>>> ----------------------
>>>> RUNNING TESTCASES
>>>>
>>>> Run 'mvn test -Dtest=TESTCASECLASS'
>>>>
>>>> To run multiple testcases separate the testcases name with comma
>>>>
>>>> To run all testcases don't specifiy -'Dtest=...'
>>>>
>>>> NOTE: TESTCASECLASS is just the testcase classname, no package name, no
>>>> extension.
>>>>
>>>> ----------------------
>>>> CREATING THE TAR
>>>>
>>>> Run 'mvn package -Pbintar -DskipTests'
>>>>
>>>> NOTE: The '-Ptar' profile will create the legacy layout, but the Hadoop
>>>> scripts will not work with the legacy layout (this has been the case
>>> before
>>>>  HADOOP-6671)
>>>>
>>>> ----------------------
>>>> RUNNING THE HADOOP SCRIPTS IN DEVELOPMENT
>>>>
>>>> Run 'mvn package -Pbintar -DskipTests'
>>>>
>>>> The Hadoop scripts can be executed from
>>>> hadoop-common/target/hadoop-common-0.23.0-SNAPSHOT-bin/bin/ directory.
>>>>
>>>> ----------------------
>>>
>>>
>
>

Mime
View raw message