flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fabian Hueske (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-5802) Flink SQL calling Hive User-Defined Functions
Date Wed, 15 Feb 2017 10:19:41 GMT

    [ https://issues.apache.org/jira/browse/FLINK-5802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15867592#comment-15867592

Fabian Hueske commented on FLINK-5802:

Hi [~clarkyzl], sure. I gave you contributor permissions and assigned the issue to you. You
can now also assign issues to yourself.

I would suggest to split this issue into four issues, one for each Hive UDF, i.e., create
new JIRA issues and make them subissues of this one.

> Flink SQL calling Hive User-Defined Functions
> ---------------------------------------------
>                 Key: FLINK-5802
>                 URL: https://issues.apache.org/jira/browse/FLINK-5802
>             Project: Flink
>          Issue Type: New Feature
>          Components: Table API & SQL
>            Reporter: Zhuoluo Yang
>            Assignee: Zhuoluo Yang
>              Labels: features
> It's important to call hive udf in Flink SQL. A great many udfs were written in hive
since last ten years. 
> It's really important to reuse the hive udfs. This feature will reduce the cost of migration
and bring more users to flink.
> Spark SQL has already supported this function.
> https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.0/bk_spark-guide/content/calling-udfs.html
> The Hive UDFs here include both built-in UDFs and customized UDFs. As many business logic
had been written in UDFs, the customized UDFs are more important than the built-in UDFs. 
> Generally, there are three kinds of UDFs in Hive: UDF, UDTF and UDAF.
> Here is the document of the Spark SQL: http://spark.apache.org/docs/latest/sql-programming-guide.html#compatibility-with-apache-hive

> Spark code:
> https://github.com/apache/spark/blob/master/sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUDFs.scala
> https://github.com/apache/spark/blob/master/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveInspectors.scala

This message was sent by Atlassian JIRA

View raw message