Mailing-List: contact user-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hive.apache.org
Received-SPF: pass (athena.apache.org: domain of edlinuxguru@gmail.com
 designates 209.85.161.176 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <A8260C5F5DD89247B5378BF1AC15B05779E1136A@Mail-srv.pursway.local>
References: <A8260C5F5DD89247B5378BF1AC15B05779E1136A@Mail-srv.pursway.local>
Date: Tue, 29 May 2012 09:57:54 -0400
Message-ID: 
 <CAENxBwzLfXtHGyOJd_8nGO3BgVZuVUGbec7bBySura3n424SvQ@mail.gmail.com>
Subject: Re: GenericUdf and Jdbc issues
From: Edward Capriolo <edlinuxguru@gmail.com>
To: user@hive.apache.org
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

So.....
this.getResourceAsStream(filename) is a very tricky method to get
right especially in hive which you have the hive-classpath, the
hadoop-classpath, the hive-jdbc classpath. Especially when you
consider that launched map/reduce tasks get there own environment and
classpath.

I had the same issues when I was writing my geo-ip-udf. See the comments.

https://github.com/edwardcapriolo/hive-geoip/blob/master/src/main/java/com/=
jointhegrid/udf/geoip/GenericUDFGeoIP.java

I came to the conclusion that if you add a file to the distributed
cache using 'ADD FILE'
You can reliably assume it will be in the current working directory
and this works.

        File f =3D new File(database);

I hope this helps.
Edward

On Tue, May 29, 2012 at 8:35 AM, Maoz Gelbart <Maoz.Gelbart@pursway.com> wr=
ote:
> Hi all,
>
>
>
> I am using Hive 0.7.1 over Cloudera=92s Hadoop distribution 0.20.2 and Ma=
pR
> hdfs distribution 1.1.1.
>
> I wrote a GenericUDF packaged as a Jar that attempts to open a local
> resource during its initialization at initialize(ObjectInspector[]
> arguments) command.
>
>
>
> When I run with the CLI, everything is fine.
>
> When I run using Cloudera=92s Hive-JDBC driver, The UDF fails with null
> pointer returned from the command this.getResourceAsStream(filename).
>
> Removing the line fixed the problem and the UDF ran on both CLI and Jdbc,=
 so
> I believe that =93ADD JAR=94 and =93CREATE TEMPORARY FUNCTION=94 were ent=
ered
> correctly.
>
>
>
> Did anyone observe such a behavior? I have a demo Jar to reproduce the
> problem if needed.
>
>
>
> Thanks,
>
> Maoz