hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Edward Capriolo <edlinuxg...@gmail.com>
Subject Re: UDF ;; FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.FunctionTask
Date Fri, 03 Feb 2012 01:34:31 GMT
Those are pretty crappy instructions. Best to build jars with maven or ant
or an iDE.

On Thu, Feb 2, 2012 at 8:27 PM, Time Less <timelessness@gmail.com> wrote:

> I sat down with one of our Java engineers who solved it. It actually
> appears we're being affected by a slightly-incorrect set of instructions on
> the internet. The typical web instructions (duplicated in many places on
> the web, such as here
> http://www.findnwrite.com/musings/extract-top-n-records-in-each-group-in-hadoophive/)
> say to build the .jar like so:
>
> *$ jar -cf LeoRank.jar /tmp/jar/com/riot/hive/udf/LeoRank.class
> *
> However, this creates a JAR like so:
>
> *$ jar -tf LeoRank.jar
> META-INF/
> META-INF/MANIFEST.MF
> /tmp/jar/com/riot/hive/udf/LeoRank.class
> *
> Note the class location in the JAR! But we're creating the package as
> com.riot.hive.udf, NOT as tmp.jar.com.riot.hive.udf, so it breaks. The
> solution is to change the "jar -cf" command like so:
>
> *$ cd /tmp/jar
> $ jar -cf LeoRank.jar ./com/riot/hive/udf/LeoRank.class
> $ jar -tf LeoRank.jar
> META-INF/
> META-INF/MANIFEST.MF
> com/riot/hive/udf/LeoRank.class
> *
> Note the now-proper class location in the JAR.
>
> The error is quite generic (read the wide variety of advices given to
> people getting this error, from "set your classpath" to "name your function
> correctly"). Since the commonly-referenced web instructions for this seem
> to lead people astray, hopefully this post will help others in the future.
>
> --
> Tim Ellis
> Data Architect, Riot Games
>
>
> On Thu, Feb 2, 2012 at 5:08 PM, Edward Capriolo <edlinuxguru@gmail.com>wrote:
>
>> Maybe you are being effected by this?
>>
>> https://issues.apache.org/jira/browse/HIVE-2635
>>
>>
>>
>>
>> On Thu, Feb 2, 2012 at 6:38 PM, Time Less <timelessness@gmail.com> wrote:
>>
>>> I'm having a heck of a time the past couple days. Google suggests others
>>> have had this same error without resolution since mid-2010. Maybe someone
>>> here can shed some light on this?
>>>
>>> *See the package and method name:
>>> *[hdfs@laxhadoop1-012 15:23:14 ~/Tim] :) head LeoRank.java
>>> package com.riot.hive.udf;
>>> import org.apache.hadoop.hive.ql.exec.UDF;
>>> public final class *LeoRank* extends UDF{
>>>
>>> *We set the classpath and ensure .jar is there:
>>> *[hdfs@laxhadoop1-012 15:24:35 ~/Tim] :) export
>>> HADOOP_CLASSPATH=/usr/lib/hive/lib
>>> [hdfs@laxhadoop1-012 15:24:44 ~/Tim] :) ls -al
>>> $HADOOP_CLASSPATH/LeoRank.jar
>>> -rw-rw-r-- 1 hdfs hdfs 884 Feb  2 15:07 /usr/lib/hive/lib/LeoRank.jar
>>>
>>> *We add the jar:
>>> *[hdfs@laxhadoop1-012 15:24:52 ~/Tim] :) hive
>>> Hive history file=/tmp/hdfs/hive_job_log_hdfs_201202021524_1137940055.txt
>>> hive> add jar LeoRank.jar ;
>>> Added LeoRank.jar to class path
>>> Added resource: LeoRank.jar
>>>
>>> *And double-check the .jar is loaded:
>>> *hive> list jars ;
>>> LeoRank.jar
>>>
>>> *And yet... fail:*
>>> hive> create temporary function LeoRank as 'com.riot.hive.udf.LeoRank' ;
>>> FAILED: Execution Error, return code 1 from
>>> org.apache.hadoop.hive.ql.exec.FunctionTask
>>>
>>> *hive.log doesn't seem to shed any additional light on it (or does it?):
>>> *
>>> [hdfs@laxhadoop1-012 15:31:16 ~/Tim] :) tail -f /tmp/hdfs/hive.log
>>> 2012-02-02 15:31:13,087 ERROR ql.Driver
>>> (SessionState.java:printError(365)) - FAILED: Execution Error, return code
>>> 1 from org.apache.hadoop.hive.ql.exec.FunctionTask
>>>
>>> There are posts on forums through history where people go through
>>> exactly this process, get to exactly this point, then nothing. I'm not sure
>>> if they give up on UDFs, or if they have some brain flash around a
>>> configuration they forgot to set or what. Any clues for the gamers of the
>>> world?
>>>
>>> --
>>> Tim Ellis
>>> Data Architect, Riot Games
>>>
>>>
>>
>

Mime
View raw message