flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-3849) Add FilterableTableSource interface and translation rule
Date Wed, 15 Mar 2017 12:15:42 GMT

    [ https://issues.apache.org/jira/browse/FLINK-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15926034#comment-15926034
] 

ASF GitHub Bot commented on FLINK-3849:
---------------------------------------

Github user godfreyhe commented on a diff in the pull request:

    https://github.com/apache/flink/pull/3520#discussion_r106149747
  
    --- Diff: flink-libraries/flink-table/src/main/scala/org/apache/flink/table/sources/FilterableTableSource.scala
---
    @@ -34,14 +34,24 @@ trait FilterableTableSource {
     
       /**
         * Check and pick all predicates this table source can support. The passed in predicates
    -    * have been translated in conjunctive form, and table source and only pick those
predicates
    -    * that it supports. All unsupported predicates should be give back to the framework
to do
    -    * further filtering.
    +    * have been translated in conjunctive form, and table source can only pick those
predicates
    +    * that it supports.
    +    * <p>
    +    * After trying to push predicates down, we should return a new [[FilterableTableSource]]
    +    * instance which holds all pushed down predicates. Even if we actually pushed nothing
down,
    +    * it is recommended that we still return a new [[FilterableTableSource]] instance
since we will
    +    * mark the returned instance as filter push down has been tried. Also we need to
return all
    +    * unsupported predicates back to the framework to do further filtering.
    +    * <p>
    +    * We also should note to not changing the form of the predicates passed in. It has
been
    +    * organized in CNF conjunctive form, and we should only take or leave each element
in the
    +    * array. Don't try to reorganize the predicates if you are absolutely confident with
that.
         *
         * @param predicate An array contains conjunctive predicates.
    -    * @return An array contains all unsupported predicates.
    +    * @return A new cloned instance of [[FilterableTableSource]] as well as n array of
Expression
    +    *         which contains all unsupported predicates.
         */
    -  def applyPredicate(predicate: Array[Expression]): Array[Expression]
    +  def applyPredicate(predicate: Array[Expression]): (FilterableTableSource[_], Array[Expression])
    --- End diff --
    
    return `FilterableTableSource[T]` instead of `FilterableTableSource[_]`, otherwise in
java, table source instance must be cast to `FilterableTableSource<?>`


> Add FilterableTableSource interface and translation rule
> --------------------------------------------------------
>
>                 Key: FLINK-3849
>                 URL: https://issues.apache.org/jira/browse/FLINK-3849
>             Project: Flink
>          Issue Type: New Feature
>          Components: Table API & SQL
>            Reporter: Fabian Hueske
>            Assignee: Kurt Young
>
> Add a {{FilterableTableSource}} interface for {{TableSource}} implementations which support
filter push-down.
> The interface could look as follows
> {code}
> def trait FilterableTableSource {
>   // returns unsupported predicate expression
>   def setPredicate(predicate: Expression): Expression
> }
> {code}
> In addition we need Calcite rules to push a predicate (or parts of it) into a TableScan
that refers to a {{FilterableTableSource}}. We might need to tweak the cost model as well
to push the optimizer in the right direction.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message