hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anant Nag (JIRA)" <>
Subject [jira] [Commented] (HIVE-9664) Hive "add jar" command should be able to download and add jars from a repository
Date Thu, 26 Feb 2015 11:52:04 GMT


Anant Nag commented on HIVE-9664:

[~appodictic], [~leftylev]: Thank you for your input. I'm making use of the grapes internally
to download resources from the artifactory. 

Following are the changes that will be made to the commands: 

Add resource command:

Adding resources from hdfs and local filesystem works as before. Resources can be added from
the artifactory using the following: 

add jar <ivy://org:module:version?query_string> <jar ivy://org:module:version?query_string>*

group - Which module group the module comes from. Translates directly to a Maven groupId or
an Ivy Organization. 
module - The name of the module to load. Translated directly to a Maven artifactId or an Ivy
version - The version of the module to use. Any version or * (for latest) or an Ivy Range
'[2.2.1,)' meaning 2.2.1 or any greater version) can be used.

Various parameters can be passed in the url to configure how and which jars are added to the
artifactory. The parameters are in the form of key value pairs where key is the name of the
paramter and the value is the value of the parameter. These key-value pairs should be separated
by &.

add jar ivy://org:module:version?key=value&key1=value1&key2=value2;

The different parameters that can be passed are:

1) exclude: Takes a comma separated value of the form (org:module)
Sometimes you will want to exclude transitive dependencies as you might be already using a
slightly different but compatible version of some artifact.  exclude can be defined in the
query string. It takes a comma(,) separated values of the form org:module, all these dependencies
won't be downloaded from the artifactory.

add jar ivy://org:module:version?exclude=org1:module1,org2:module2,org3:module3;
add jar ivy://org.apache.pig:pig:0.10.0?exclude=org.apache.hadoop:avro;
2) transitive: Takes values true or false. Defaults to true.
When transitive = true, all the transitive dependencies are downloaded and added to the classpath.

add jar ivy://org:module:version?transitive=true&exclude=org1:module1
add jar ivy://org.module:version?transitive=false;

add jar ivy://org.apache.pig:pig:0.10.0?exclude=org.apache.hadoop:avro&transitive=true;
add jar ivy://org.apache.pig:pig:0.10.0?transitive=false

3) ext: The extension of the file to add. jar by default.

add jar ivy://org.apache.pig:pig:0.10.0?ext=jar
add jar ivy://com.linkedin.informed-bridge-jira:informed-bridge-jira:0.0.3?classifier=docs-json&ext=tar.gz;

4) classifier: The maven classifier to resolve by

add jar ivy://com.linkedin.informed-bridge-jira:informed-bridge-jira:0.0.3?classifier=docs-json&ext=tar.gz;

Delete resource command:

Delete resource works as before for the resources added from hdfs and local file system.
Resources added from the artifactory can be deleted using the following command.

delete jar ivy://org:module:version

The delete jar command will delete all the transitive dependencies of the jar which was added
using the same org:module:version. 

If two jars share a set of transitive dependencies and one of the jars is deleted using the
above syntax, then all the transitive dependencies will be deleted for the jar except the
ones which are shared. 

add jar ivy://org.apache.pig:pig:0.10.0
add jar ivy://org.apache.pig:pig:
delete jar ivy://org.apache.pig:pig:0.10.0

if A is the set containing the transitive dependencies of pig-0.10.0 and B is the set containing
the transitive dependencies of pig-, then after executing the above commands, A-(A
intersection B) will be deleted. 

Now on executing
delete jar ivy://org.apache.pig:pig:
all the remaining dependencies will be deleted. 

List resource: works as before. 

> Hive "add jar" command should be able to download and add jars from a repository
> --------------------------------------------------------------------------------
>                 Key: HIVE-9664
>                 URL:
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Anant Nag
>              Labels: hive
> Currently Hive's "add jar" command takes a local path to the dependency jar. This clutters
the local file-system as users may forget to remove this jar later
> It would be nice if Hive supported a Gradle like notation to download the jar from a
> Example:  add jar org:module:version
> It should also be backward compatible and should take jar from the local file-system
as well. 

This message was sent by Atlassian JIRA

View raw message