flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-8492) [FLINK-8492][table] Fix calc cost bug
Date Thu, 25 Jan 2018 15:58:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-8492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16339419#comment-16339419
] 

ASF GitHub Bot commented on FLINK-8492:
---------------------------------------

Github user fhueske commented on a diff in the pull request:

    https://github.com/apache/flink/pull/5347#discussion_r163880994
  
    --- Diff: flink-libraries/flink-table/src/main/scala/org/apache/flink/table/plan/rules/common/CorrelateUtil.scala
---
    @@ -0,0 +1,79 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one
    + * or more contributor license agreements.  See the NOTICE file
    + * distributed with this work for additional information
    + * regarding copyright ownership.  The ASF licenses this file
    + * to you under the Apache License, Version 2.0 (the
    + * "License"); you may not use this file except in compliance
    + * with the License.  You may obtain a copy of the License at
    + *
    + *     http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +package org.apache.flink.table.plan.rules.common
    +
    +import org.apache.calcite.plan.volcano.RelSubset
    +import org.apache.calcite.rel.RelNode
    +import org.apache.calcite.rex.{RexProgram, RexProgramBuilder}
    +import org.apache.flink.table.api.TableException
    +import org.apache.flink.table.plan.nodes.logical.{FlinkLogicalCalc, FlinkLogicalTableFunctionScan}
    +
    +/**
    +  * An utility for datasteam and dataset correlate rules.
    +  */
    +object CorrelateUtil {
    +
    +  /**
    +    * Find only calc and table function
    +    */
    +  def findCalcAndTableFunction(calc: FlinkLogicalCalc): Boolean = {
    +    val child = calc.getInput.asInstanceOf[RelSubset].getOriginal
    +    child match {
    +      case scan: FlinkLogicalTableFunctionScan => true
    +      case calc: FlinkLogicalCalc => findCalcAndTableFunction(calc)
    +      case _ => false
    +    }
    +  }
    +
    +  /**
    +    * Get [[FlinkLogicalTableFunctionScan]] from the input calc.
    +    */
    +  def getTableScan(calc: FlinkLogicalCalc): RelNode = {
    --- End diff --
    
    rename to `getTableFunctionScan`


> [FLINK-8492][table] Fix calc cost bug
> -------------------------------------
>
>                 Key: FLINK-8492
>                 URL: https://issues.apache.org/jira/browse/FLINK-8492
>             Project: Flink
>          Issue Type: Bug
>          Components: Table API &amp; SQL
>            Reporter: Hequn Cheng
>            Assignee: Hequn Cheng
>            Priority: Major
>
> Considering the following test, unsupported exception will be thrown due to multi
calc existing between correlate and TableFunctionScan.
> {code:java}
> // code placeholder
> @Test
> def testCrossJoinWithMultiFilter(): Unit = {
>   val t = testData(env).toTable(tEnv).as('a, 'b, 'c)
>   val func0 = new TableFunc0
>   val result = t
>     .join(func0('c) as('d, 'e))
>     .select('c, 'd, 'e)
>     .where('e > 10)
>     .where('e > 20)
>     .select('c, 'd)
>     .toAppendStream[Row]
>   result.addSink(new StreamITCase.StringSink[Row])
>   env.execute()
>   val expected = mutable.MutableList("Jack#22,Jack,22", "Anna#44,Anna,44")
>   assertEquals(expected.sorted, StreamITCase.testResults.sorted)
> }
> {code}
> I can see two options to fix this problem:
>  # Adapt calcite OptRule to merge the continuous calc.
>  # Merge multi calc in correlate convert rule.
> I prefer the second one, not only it is easy to implement but also i think with or without
an optimize rule should not influence flink functionality. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message