hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Giridharan Kesavan (JIRA)" <j...@apache.org>
Subject [jira] Issue Comment Edited: (HADOOP-3305) Publish hadoop-core to the apache repository with an appropriate POM file
Date Mon, 24 Nov 2008 14:06:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12650202#action_12650202
] 

gkesavan edited comment on HADOOP-3305 at 11/24/08 6:06 AM:
----------------------------------------------------------------------

This patch *HADOOP-3305.patch* is developed from the patch submitted by Steve, Thanks to Steve.
*rmlib.sh* is the supporting script to cleanup the jar file from the lib folder.
 
With this patch we should be able to use ivy to resolve dependencies through the maven repository,
and we can get rid of the local lib folder when all the dependencies all available on the
m2 repository.

As Steve mentioned there are certain missing libraries in the m2 repo. 

Those missing dependencies are still added to the classpath from the local lib folder, and
other dependencies that are available in the m2 repository are resolved/retrieved by IVY from
the maven repository.

Here I 've provided the list of missing dependencies for different component.

*Hadoop-core*
-commons-cli-2.0-SNAPSHOT.jar  - 20040117.000000 snapshot available..
-kfs-0.2.2.jar		                              - NOT AVAILABLE	

*Thrift*
-hadoopthriftapi.jar     -    NOT AVAILABLE	
-libthrift.jar                    -    NOT AVAILABLE	

*Chukwa*
-json.jar       -              Not sure fo the jar file version though json is available in
the m2 respo.

This patch contains a new set of ivybuild.xml & ivy.xml files for diff comps 
which resembles the existing buid.xml files except for the ivy implementations. 
The ivy.xml file has all the dependencies for diff components listed in it with versions
pulled out from the corresponding components libraries.properties file.

Also this patch contains a top level ivysettings.xml file which has the details of all the
resolvers and the url for resolving the dependencies.


(on) 
*Instruction to test the patch*

Apply the patch
run the rmlib.sh script from hadoop-core base folder. 
_[ This would remove the local lib folder with jar files that can be resolved/retrieved from
the m2 repository]_

As of now the patch supports *3 main targets*

*compile
*package
*releaseaudit

To execute the targets 
{quote}
ant -f ivybuild.xml releaseaudit 
{quote}
This has compile and package targets as dependencies.

Work in progress for other ant targets. Meanwhile I would like to get this patch reviewed.
Please review this patch.

Thanks,
Giri

      was (Author: gkesavan):
    This patch is developed from the patch submitted by Steve, Thanks to Steve.

With this patch we should be able to use ivy to resolve dependencies through the maven repository,
and we can get rid of the local lib folder when all the dependencies all available on the
m2 repository.

As Steve mentioned there are certain missing libraries in the m2 repo. 

Those missing dependencies are still added to the classpath from the local lib folder, and
other dependencies that are available in the m2 repository are resolved/retrieved by IVY from
the maven repository.

Here I 've provided the list of missing dependencies for different component.

*Hadoop-core*
-commons-cli-2.0-SNAPSHOT.jar  - 20040117.000000 snapshot available..
-kfs-0.2.2.jar		                              - NOT AVAILABLE	

*Thrift*
-hadoopthriftapi.jar     -    NOT AVAILABLE	
-libthrift.jar                    -    NOT AVAILABLE	

*Chukwa*
-json.jar       -              Not sure fo the jar file version though json is available in
the m2 respo.

This patch contains a new set of ivybuild.xml & ivy.xml files for diff comps 
which resembles the existing buid.xml files except for the ivy implementations. 
The ivy.xml file has all the dependencies for diff components listed in it with versions
pulled out from the corresponding components libraries.properties file.

Also this patch contains a top level ivysettings.xml file which has the details of all the
resolvers and the url for resolving the dependencies.
 
*Instruction to test the patch*

Apply the patch
run the rmlib.sh script from hadoop-core base folder. 
_[ This would remove the local lib folder with jar files that can be resolved/retrieved from
the m2 repository]_

As of now the patch supports *3 main targets*

*compile
*package
*releaseaudit

To execute the targets 
{quote}
ant -f ivybuild.xml releaseaudit 
{quote}
This has compile and package targets as dependencies.

Work in progress for other ant targets. Meanwhile I would like to get this patch reviewed.
Please review this patch.

Thanks,
Giri
  
> Publish hadoop-core to the apache repository with an appropriate POM file
> -------------------------------------------------------------------------
>
>                 Key: HADOOP-3305
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3305
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: build
>    Affects Versions: 0.16.2, 0.16.3
>            Reporter: Steve Loughran
>            Priority: Minor
>         Attachments: HADOOP-3305.patch, hadoop-core-0.16.2.pom, ivy-support-first-pass.zip,
rmlib.sh
>
>
> To let people downstream build/test with hadoop, using Apache Ivy or Apache Maven2 to
pull it down, hadoop-core needs to be published to the apache repository with a .pom file
that lists its mandatory dependencies.
> In an automated build process, this means
> -having a template XML pom defining all included dependencies (and excluded transient
dependency artifacts)
> -having a property file driving version numbering of all artifacts
> -copying this template with property expansion to create the release POM file
> -public releases only: sticking this POM file up on people.apache.org in the right place,
along with the JAR and some .md5 checksums
> There's a risk that if the hadoop team dont do this, someone else will (as mahout are
doing under http://people.apache.org/~kalle/mahout/maven2/org/apache/hadoop/ )
> This is bad as hadoop end up fielding the support calls from someone elses files. 
> Before automating the process, existing hadoop-core JARs can be pushed out with hand-encoded
POM files. The repository police dont allow pom files ever to be changed, so supporting existing
releases (.16.2, 0.16.3 ... ) is a way of beta testing the POMs. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message