flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bowen Li (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-3089) State API Should Support Data Expiration (State TTL)
Date Wed, 04 Oct 2017 04:18:01 GMT

    [ https://issues.apache.org/jira/browse/FLINK-3089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16190781#comment-16190781
] 

Bowen Li commented on FLINK-3089:
---------------------------------

[~srichter] Do you mean that TTL doesn't guarantee a record is immediately removed after the
'time' passes, and we don't know if that record is checkpointed or not if Flink takes a snapshot
shortly after that 'time'?

> State API Should Support Data Expiration (State TTL)
> ----------------------------------------------------
>
>                 Key: FLINK-3089
>                 URL: https://issues.apache.org/jira/browse/FLINK-3089
>             Project: Flink
>          Issue Type: New Feature
>          Components: DataStream API, State Backends, Checkpointing
>            Reporter: Niels Basjes
>
> In some usecases (webanalytics) there is a need to have a state per visitor on a website
(i.e. keyBy(sessionid) ).
> At some point the visitor simply leaves and no longer creates new events (so a special
'end of session' event will not occur).
> The only way to determine that a visitor has left is by choosing a timeout, like "After
30 minutes no events we consider the visitor 'gone'".
> Only after this (chosen) timeout has expired should we discard this state.
> In the Trigger part of Windows we can set a timer and close/discard this kind of information.
But that introduces the buffering effect of the window (which in some scenarios is unwanted).
> What I would like is to be able to set a timeout on a specific state which I can update
afterwards.
> This makes it possible to create a map function that assigns the right value and that
discards the state automatically.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message