flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-9681) Make sure minRetentionTime not equal to maxRetentionTime
Date Thu, 05 Jul 2018 12:58:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-9681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533618#comment-16533618
] 

ASF GitHub Bot commented on FLINK-9681:
---------------------------------------

Github user fhueske commented on a diff in the pull request:

    https://github.com/apache/flink/pull/6255#discussion_r200330683
  
    --- Diff: docs/dev/table/streaming.md ---
    @@ -591,16 +589,14 @@ qConfig.withIdleStateRetentionTime(Time.hours(12);
     
     val qConfig: StreamQueryConfig = ???
     
    -// set idle state retention time: min = 12 hour, max = 16 hours
    -qConfig.withIdleStateRetentionTime(Time.hours(12), Time.hours(16))
    -// set idle state retention time. min = max = 12 hours
    -qConfig.withIdleStateRetentionTime(Time.hours(12)
    +// set idle state retention time: min = 12 hour, max = 24 hours
    +qConfig.withIdleStateRetentionTime(Time.hours(12), Time.hours(24))
     
     {% endhighlight %}
     </div>
     </div>
     
    -Configuring different minimum and maximum idle state retention times is more efficient
because it reduces the internal book-keeping of a query for when to remove state.
    +Configuring different minimum and maximum idle state retention times is more efficient
because it reduces the internal book-keeping of a query for when to remove state. Difference
between minTime and maxTime shoud be at least 5 minutes.
    --- End diff --
    
    The "... more efficient ..." does not apply anymore. Maybe rephrase to 
    
    > Cleaning up state requires additional bookkeeping which becomes less expensive for
larger differences of `minTime` and `maxTime`. The difference between `minTime` and `maxTime`
must be at least 5 minutes.
    



> Make sure minRetentionTime not equal to maxRetentionTime
> --------------------------------------------------------
>
>                 Key: FLINK-9681
>                 URL: https://issues.apache.org/jira/browse/FLINK-9681
>             Project: Flink
>          Issue Type: Improvement
>          Components: Table API &amp; SQL
>            Reporter: Hequn Cheng
>            Assignee: Hequn Cheng
>            Priority: Major
>              Labels: pull-request-available
>
> Currently, for a group by(or other operators), if minRetentionTime equals to maxRetentionTime,
the group by operator will register a timer for each record coming at different time which
cause performance problem. The reasoning for having two parameters is that we can avoid to
register many timers if we have more freedom when to discard state. As min equals to max cause
performance problem it is better to make sure these two parameters are not same.
> Any suggestions are welcome.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message