hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joydeep Sen Sarma (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-322) cannot create temporary udf dynamically, with a ClassNotFoundException
Date Tue, 10 Mar 2009 03:16:50 GMT

    [ https://issues.apache.org/jira/browse/HIVE-322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12680368#action_12680368

Joydeep Sen Sarma commented on HIVE-322:

for the thrift server mode -  where does the jar file sit? if it is available on the client
side - doesn't it need to be transferred to the server side via the Thrift interface? 

the set commands are also not available through the Thrift server - and i agree that all of
these should be accessible via the Thrift interface (ignoring the file location issue). I
think it should be fairly easy to refactor Thrift server and Driver/CliDriver code to make
sure that all commands are supported on the ThriftServer (and don't think that we need to
bring all the cli commands into the Hive grammar for that to happen).

let's open another jira for this issue (executing cli commands into thrift server) - it should
be fairly easy to get in. if u can add the 'add jar' command to the cli in the meantime -
we should be good to go. I looked at the patch and the main code looks good.

> cannot create temporary udf dynamically, with a ClassNotFoundException 
> -----------------------------------------------------------------------
>                 Key: HIVE-322
>                 URL: https://issues.apache.org/jira/browse/HIVE-322
>             Project: Hadoop Hive
>          Issue Type: Bug
>          Components: Query Processor
>    Affects Versions: 0.3.0
>            Reporter: Min Zhou
>            Priority: Blocker
>         Attachments: registerjars-v3.patch, registerjars1.patch, registerjars2.patch
> I found the ClassLoader cannot load my UDF when doing FunctionTask, because the ClassLoader
hasnot append its classpaths on-the-fly yet.
> The ExecDriver' s addToClassPath(String[] newPaths) method is the only entry for ClassLoader
dynamically append its classhpaths (besides hadoop's GenericOptionsParser).
> But that function wasnot called before FunctionTask getting my UDF class by class name.
I think this is the reason why I came across that failure.
> scenario description:
> I set a peroperty in hive-site.xml to configure the classpath of my udf. 
> <property>
>   <name>hive.aux.jars.path</name>
>   <value>/home/hadoop/hdpsoft/hive-auxs/zhoumin.jar</value>
> </property>
> but failed to register it with a ClassNotFoundException when creating udf through the
sql command.
> CREATE TEMPORARY FUNCTION strlen AS 'hadoop.hive.udf.UdfStringLength'
> I'll make a patch soon.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message