hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Kerzner <mark.kerz...@shmsoft.com>
Subject Re: Building dependencies into UDAF jar
Date Mon, 07 Nov 2011 19:45:49 GMT
Here is an example of assembly plugin,
https://github.com/markkerzner/HadoopInPracticeCode/blob/master/pom.xml

Mark

On Mon, Nov 7, 2011 at 1:38 PM, Frank Maritato <fmaritato@attinteractive.com
> wrote:

>   I'm not sure how to do this in ant, but it definitely works with
> maven+assembly because this is what we do for our production map reduce
> jobs and for our hive udf's. If you want I can probably dig up an example
> pom.xml that shows how to do this.
>  --
> Frank Maritato
>
>   From: Pradeep Kamath <pradeepkth@yahoo.com>
> Reply-To: <user@hive.apache.org>, Pradeep Kamath <pradeepkth@yahoo.com>
> Date: Mon, 7 Nov 2011 11:25:15 -0800
> To: "user@hive.apache.org" <user@hive.apache.org>
> Subject: Re: Building dependencies into UDAF jar
>
>   Thanks for the suggestion Frank. Unfortunately I am using ant and am
> not very familiar with maven. Glancing through the link on
> maven-assembly-plugin, I am also not sure if the "assembly" created will
> work fine with hive/hadoop when the task runs on a remote node. Any way of
> creating a jar which hive/hadoop will understand to be a "composite" jar
> containing main classes and other dependent jars?
>
>   ------------------------------
> *From:* Frank Maritato <fmaritato@attinteractive.com>
> *To:* "user@hive.apache.org" <user@hive.apache.org>; Pradeep Kamath <
> pradeepkth@yahoo.com>
> *Sent:* Monday, November 7, 2011 10:47 AM
> *Subject:* Re: Building dependencies into UDAF jar
>
>   Hi Pradeep,
>
>  If you are using maven to build your code you can use the assembly
> plugin to produce a jar with dependencies.
> http://maven.apache.org/plugins/maven-assembly-plugin/
>
>  Hope this helps
>  --
> Frank Maritato
>
>   From: Pradeep Kamath <pradeepkth@yahoo.com>
> Reply-To: <user@hive.apache.org>, Pradeep Kamath <pradeepkth@yahoo.com>
> Date: Mon, 7 Nov 2011 10:21:26 -0800
> To: "user@hive.apache.org" <user@hive.apache.org>
> Subject: Building dependencies into UDAF jar
>
>   Hi,
>    I am trying to build a hive UDAF which has dependencies on other
> external jars (like log4j) - is there a way in which I can bundle the main
> classes for my UDAF and all the dependent jars into one jar for use in my
> hive query. I tried having my UDF classes at the top level and dependency
> jars in a "lib" directory inside one jar and then using "add jar UDAF.jar"
> but that did not work (classes from dependency jars were not being
> resolved). Setting HIVE_AUX_JARS_PATH environment variable does work but
> requires that I have all the dependent jars separately on the local
> filesystem where I run the hive query from. It would be more convenient for
> me to build one self contained jar and refer to that in my query (due to
> some restrictions in the cluster I run the query in) - is that possible?
>
>  Thanks,
> Pradeep
>
>
>

Mime
View raw message