flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-8492) [FLINK-8492][table] Fix calc cost bug
Date Thu, 25 Jan 2018 12:00:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-8492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16339126#comment-16339126
] 

ASF GitHub Bot commented on FLINK-8492:
---------------------------------------

Github user hequn8128 commented on the issue:

    https://github.com/apache/flink/pull/5347
  
    hi @fhueske @twalthr , the pr has been updated. It would be great if you can take a look
at it.
    
    Changes mainly include: 
    1. Adapt estimateRowCount to be more accurate. The original implementation use a constant
0.75 to reduce the result which makes row count of merged calc bigger than the row count of
un-merged calcs. Current implementation use a more accurate selectivity to reduce the result
row count.
    2. Merge calcs in convert rule of correrate. Double check to make sure unsupported exception
won't be thrown


> [FLINK-8492][table] Fix calc cost bug
> -------------------------------------
>
>                 Key: FLINK-8492
>                 URL: https://issues.apache.org/jira/browse/FLINK-8492
>             Project: Flink
>          Issue Type: Bug
>          Components: Table API &amp; SQL
>            Reporter: Hequn Cheng
>            Assignee: Hequn Cheng
>            Priority: Major
>
> Considering the following test, unsupported exception will be thrown due to multi
calc existing between correlate and TableFunctionScan.
> {code:java}
> // code placeholder
> @Test
> def testCrossJoinWithMultiFilter(): Unit = {
>   val t = testData(env).toTable(tEnv).as('a, 'b, 'c)
>   val func0 = new TableFunc0
>   val result = t
>     .join(func0('c) as('d, 'e))
>     .select('c, 'd, 'e)
>     .where('e > 10)
>     .where('e > 20)
>     .select('c, 'd)
>     .toAppendStream[Row]
>   result.addSink(new StreamITCase.StringSink[Row])
>   env.execute()
>   val expected = mutable.MutableList("Jack#22,Jack,22", "Anna#44,Anna,44")
>   assertEquals(expected.sorted, StreamITCase.testResults.sorted)
> }
> {code}
> I can see two options to fix this problem:
>  # Adapt calcite OptRule to merge the continuous calc.
>  # Merge multi calc in correlate convert rule.
> I prefer the second one, not only it is easy to implement but also i think with or without
an optimize rule should not influence flink functionality. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message