hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eugene Koifman <ekoif...@hortonworks.com>
Subject Re: UDF cannot be found when the query is submitted via templeton
Date Mon, 20 Apr 2015 14:32:16 GMT
can you give the complete REST call you are making to submit the query?

From: Xiaoyong Zhu <xiaoyzhu@microsoft.com<mailto:xiaoyzhu@microsoft.com>>
Reply-To: "user@hive.apache.org<mailto:user@hive.apache.org>" <user@hive.apache.org<mailto:user@hive.apache.org>>
Date: Sunday, April 19, 2015 at 8:23 PM
To: "user@hive.apache.org<mailto:user@hive.apache.org>" <user@hive.apache.org<mailto:user@hive.apache.org>>
Subject: RE: UDF cannot be found when the query is submitted via templeton

No, it doesn't...

What surprises me is that for HDInsight (Hadoop on Azure) which is using Azure BLOB storage,
using

ADD JAR wasb:///test/HiveUDF.jar;
CREATE TEMPORARY FUNCTION FindPat as 'HiveUDF.FindPattern'
select count(FindPat(columnname)) from table1;

would work. However, for my own cluster,

ADD JAR hdfs:///test/HiveUDF.jar;
CREATE TEMPORARY FUNCTION FindPat as 'HiveUDF.FindPattern'
select count(FindPat(columnname)) from table1;

does not work...

Xiaoyong

From: Jason Dere [mailto:jdere@hortonworks.com]
Sent: Saturday, April 18, 2015 1:37 AM
To: <user@hive.apache.org<mailto:user@hive.apache.org>>
Subject: Re: UDF cannot be found when the query is submitted via templeton

Does fully qualifying the function name (HiveUDF.FindPattern()) in the query help here?

On Apr 17, 2015, at 6:44 AM, Xiaoyong Zhu <xiaoyzhu@microsoft.com<mailto:xiaoyzhu@microsoft.com>>
wrote:


Hi experts

I am trying to use an UDF (I have already put that in the metastore using CREATE FUNCTION)
as following.

select count(FindPattern(s_sitename)) AS testcol from weblogs;

However, when I tried to use the UDF from WebHCat (i.e. submit the above command via WebHCat),
the job always fails saying

Added [/tmp/2cb22c27-72d3-4b41-aea0-655df1192872_resources/HiveUDF.jar] to class path
Added resources: [hdfs://PATHTOFOLDER/Portal-Queries/HiveUDF.jar]
FAILED: SemanticException [Error 10011]: Line 1:13 Invalid function FindPattern

If I execute this command through Hive CLI (through hive -f file or execute it in the interactive
shell) the statement above works. From the log I can see the jar file is added but it seems
the function cannot be found. Can someone help to share some thoughts on this issue?

Btw, the create function statement is as following (changing the hdfs URI to full path does
not work either):
CREATE FUNCTION FindPattern AS 'HiveUDF.FindPattern' USING JAR'hdfs:///UDFFolder/HiveUDF.jar'<hdfs://UDFFolder/HiveUDF.jar'>;

Thanks in advance!

Xiaoyong



Mime
View raw message