hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Francke (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6671) To use maven for hadoop common builds
Date Sat, 03 Apr 2010 15:03:27 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853146#action_12853146

Lars Francke commented on HADOOP-6671:

I don't have a lot of insight into the Hadoop Common project itself but I've done a lot of
the work on the recent HBase transition together with Paul Smith and Kay Kay and would like
to offer my help if needed.

There are basically two ways to go: Either shuffle around a lot of directories to conform
to the standard Maven layout or to override the Maven defaults and keep the current layout.
Both ways are actually very painful but I prefer the first one. While that means a lot of
work and a lot of swearing it also means that you'll get a consistent layout (with most other
Maven projects) and the configuration is a lot easier. Some tools tend(ed) to not work properly
with the non standard layout (strictly speaking this are bugs in the tools/plugins). You basically
have to do a lot of work up front but after that it shouldn't be too hard to maintain.

The directory moving/renaming unfortunately tends not to work too well with Subversion and
branches (or I was doing it wrong) so I don't know how big the benefit would be to start a
new one for doing this.

What Paul Smith has done in HBASE-2099 is to provide a script that contained all the necessary
commands (svn mv...) to finish the move (there are a couple more changes in other tickets)
and a patch containing the .pom files. This has been much improved since and we've learned
a lot from it. We are in fact still tweaking it and will change the pom structure once again
to rely on the common Apache parent pom in addition to a few smaller fixes that are still

I'd propose a similar structure as HBase now has: A pom in the main directory with common
information (packaging "pom") and then two modules ("core" and "contrib"). So the src/contrib
directory would move to the top-level as would the docs directory (btw: I've never used Forrest
with Maven but if Ant can do it Maven should be able to do it too). The final tarball as currently
(trunk) created by ant tar should be easily reproducible by Maven (I couldn't find where most
of the conribs ended up, are those currently missung from the .tar?)

Paul Smith has indicated interest in this and I'd be interested and willing to help too if
needed. But if you've already got someone to do the work I'm not going to complain :) It'd
just be a shame to duplicate work here as the moving around of stuff in SVN is painful work
if that's the way you decide to go.

> To use maven for hadoop common builds
> -------------------------------------
>                 Key: HADOOP-6671
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6671
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: build
>    Affects Versions: 0.22.0
>            Reporter: Giridharan Kesavan
> We are now able to publish hadoop artifacts to the maven repo successfully [ Hadoop-6382]
> Drawbacks with the current approach:
> * Use ivy for dependency management with ivy.xml
> * Use maven-ant-task for artifact publishing to the maven repository
> * pom files are not generated dynamically 
> To address this I propose we use maven to build hadoop-common, which would help us to
manage dependencies, publish artifacts and have one single xml file(POM) for dependency management
and artifact publishing.
> I would like to have a branch created to work on mavenizing  hadoop common.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message