Return-Path: <issues-return-68878-archive-asf-public=cust-asf.ponee.io@flink.apache.org>
X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io
Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io
Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183])
	by cust-asf2.ponee.io (Postfix) with ESMTP id 59450200B76
	for <archive-asf-public-internal@cust-asf2.ponee.io>; Tue, 30 Aug 2016 11:15:22 +0200 (CEST)
Received: by cust-asf.ponee.io (Postfix)
	id 57A6A160AC5; Tue, 30 Aug 2016 09:15:22 +0000 (UTC)
Delivered-To: archive-asf-public@cust-asf.ponee.io
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by cust-asf.ponee.io (Postfix) with SMTP id A11B2160AA8
	for <archive-asf-public@cust-asf.ponee.io>; Tue, 30 Aug 2016 11:15:21 +0200 (CEST)
Received: (qmail 62252 invoked by uid 500); 30 Aug 2016 09:15:20 -0000
Mailing-List: contact issues-help@flink.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:issues-help@flink.apache.org>
List-Unsubscribe: <mailto:issues-unsubscribe@flink.apache.org>
List-Post: <mailto:issues@flink.apache.org>
List-Id: <issues.flink.apache.org>
Reply-To: dev@flink.apache.org
Delivered-To: mailing list issues@flink.apache.org
Received: (qmail 62225 invoked by uid 99); 30 Aug 2016 09:15:20 -0000
Received: from arcas.apache.org (HELO arcas) (140.211.11.28)
    by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 30 Aug 2016 09:15:20 +0000
Received: from arcas.apache.org (localhost [127.0.0.1])
	by arcas (Postfix) with ESMTP id 8FB632C1B75
	for <issues@flink.apache.org>; Tue, 30 Aug 2016 09:15:20 +0000 (UTC)
Date: Tue, 30 Aug 2016 09:15:20 +0000 (UTC)
From: "Timo Walther (JIRA)" <jira@apache.org>
To: issues@flink.apache.org
Message-ID: <JIRA.12999592.1472032152000.445232.1472548520584@Atlassian.JIRA>
In-Reply-To: <JIRA.12999592.1472032152000@Atlassian.JIRA>
References: <JIRA.12999592.1472032152000@Atlassian.JIRA> <JIRA.12999592.1472032152757@arcas>
Subject: [jira] [Commented] (FLINK-4469) Add support for user defined table
 function in Table API & SQL
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394
archived-at: Tue, 30 Aug 2016 09:15:22 -0000


    [ https://issues.apache.org/jira/browse/FLINK-4469?page=3Dcom.atlassian=
.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D1544=
8532#comment-15448532 ]=20

Timo Walther commented on FLINK-4469:
-------------------------------------

Yes, this looks very good. Looking forward to reviewing a PR.

> Add support for user defined table function in Table API & SQL
> --------------------------------------------------------------
>
>                 Key: FLINK-4469
>                 URL: https://issues.apache.org/jira/browse/FLINK-4469
>             Project: Flink
>          Issue Type: New Feature
>          Components: Table API & SQL
>            Reporter: Jark Wu
>            Assignee: Jark Wu
>
> Normal user-defined functions, such as concat(), take in a single input r=
ow and output a single output row. In contrast, table-generating functions =
transform a single input row to multiple output rows. It is very useful in =
some cases, such as look up in HBase by rowkey and return one or more rows.
> Adding a user defined table function should:
> 1. inherit from UDTF class with specific generic type T
> 2. define one or more evel function.=20
> NOTE:=20
> 1. the eval method must be public and non-static.
> 2. eval should always return java.lang.Iterable or scala.collection.Itera=
ble with the generic type T.
> 3. the generic type T is the row type returned by table function. Because=
 of Java type erasure, we can=E2=80=99t extract T from the Iterable.
> 4. eval method can be overload. Blink will choose the best match eval met=
hod to call according to parameter types and number.
> {code}
> public class Word {
>   public String word;
>   public Integer length;
> }
> public class SplitStringUDTF extends UDTF<Word> {
>     public Iterable<Word> eval(String str) {
>         if (str =3D=3D null) {
>             return new ArrayList<>();
>         } else {
>             List<Word> list =3D new ArrayList<>();
>             for (String s : str.split(",")) {
>                 Word word =3D new Word(s, s.length());
>                 list.add(word);
>             }
>             return list;
>         }
>     }
> }
> // in SQL
> tableEnv.registerFunction("split", new SplitStringUDTF())
> tableEnv.sql("SELECT a, b, t.* FROM MyTable CROSS APPLY split(c) AS t(w,l=
)")
> // in Java Table API
> tableEnv.registerFunction("split", new SplitStringUDTF())
> // rename split table columns to =E2=80=9Cw=E2=80=9D and =E2=80=9Cl=E2=80=
=9D
> table.crossApply("split(c)", "w, l")=09
>      .select("a, b, w, l")
> // without renaming, we will use the origin field names in the POJO/case/=
...
> table.crossApply("split(c)")
>      .select("a, b, word, length")
> // in Scala Table API
> val split =3D new SplitStringUDTF()
> table.crossApply(split('c), 'w, 'l)
>      .select('a, 'b, 'w, 'l)
> // outerApply for outer join to a UDTF
> table.outerApply(split('c))
>      .select('a, 'b, 'word, 'length)
> {code}
> Here we introduce CROSS/OUTER APPLY keywords to join table functions , wh=
ich is used in SQL Server. We can discuss the API in the comment.=20
> Maybe the {{UDTF}} class should be replaced by {{TableFunction}} or somet=
hing others, because we have introduced {{ScalarFunction}} for custom funct=
ions, we need to keep consistent. Although, I prefer {{UDTF}} rather than {=
{TableFunction}} as the former is more SQL-like and the latter maybe confus=
ed with DataStream functions.=20
> **This issue is blocked by CALCITE-1309, so we need to wait Calcite fix t=
his and release.**
> See [1] for more information about UDTF design.
> [1] https://docs.google.com/document/d/15iVc1781dxYWm3loVQlESYvMAxEzbbuVF=
PZWBYuY1Ek/edit#



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)