spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dongjoon Hyun (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-18832) Spark SQL: Thriftserver unable to run a registered Hive UDTF
Date Mon, 06 Mar 2017 19:02:32 GMT

    [ https://issues.apache.org/jira/browse/SPARK-18832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15897841#comment-15897841
] 

Dongjoon Hyun commented on SPARK-18832:
---------------------------------------

Could you upload your `jar` file? It could be a problem of jar packaging.

> Spark SQL: Thriftserver unable to run a registered Hive UDTF
> ------------------------------------------------------------
>
>                 Key: SPARK-18832
>                 URL: https://issues.apache.org/jira/browse/SPARK-18832
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 2.0.0, 2.0.1, 2.0.2
>         Environment: HDP: 2.5
> Spark: 2.0.0
>            Reporter: Lokesh Yadav
>         Attachments: SampleUDTF.java
>
>
> Spark Thriftserver is unable to run a HiveUDTF.
> It throws the error that it is unable to find the functions although the function registration
succeeds and the funtions does show up in the list output by {{show functions}}.
> I am using a Hive UDTF, registering it using a jar placed on my local machine. Calling
it using the following commands:
> //Registering the functions, this command succeeds.
> {{CREATE FUNCTION SampleUDTF AS 'com.fuzzylogix.experiments.udf.hiveUDF.SampleUDTF' USING
JAR '/root/spark_files/experiments-1.2.jar';}}
> //Thriftserver is able to look up the functuion, on this command:
> {{DESCRIBE FUNCTION SampleUDTF;}}
> {quote}
> {noformat}
> Output: 
> +-----------------------------------------------------------+--+
> |                       function_desc                       |
> +-----------------------------------------------------------+--+
> | Function: default.SampleUDTF                              |
> | Class: com.fuzzylogix.experiments.udf.hiveUDF.SampleUDTF  |
> | Usage: N/A.                                               |
> +-----------------------------------------------------------+--+
> {noformat}
> {quote}
> // Calling the function: 
> {{SELECT SampleUDTF('Paris');}}
> bq. Output of the above command: Error: org.apache.spark.sql.AnalysisException: Undefined
function: 'SampleUDTF'. This function is neither a registered temporary function nor a permanent
function registered in the database 'default'.; line 1 pos 7 (state=,code=0)
> I have also tried with using a non-local (on hdfs) jar, but I get the same error.
> My environment: HDP 2.5 with spark 2.0.0
> I have attached the class file for the UDTF I am using in testing this.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message