Mailing-List: contact issues-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@flink.apache.org
Date: Thu, 17 Aug 2017 08:19:00 +0000 (UTC)
From: "Stefan Richter (JIRA)" <jira@apache.org>
To: issues@flink.apache.org
Message-ID: <JIRA.12916634.1448629370000.52296.1502957940244@Atlassian.JIRA>
In-Reply-To: <JIRA.12916634.1448629370000@Atlassian.JIRA>
References: <JIRA.12916634.1448629370000@Atlassian.JIRA> <JIRA.12916634.1448629370375@jira-lw-us.apache.org>
Subject: [jira] [Commented] (FLINK-3089) State API Should Support Data
 Expiration (State TTL)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Thu, 17 Aug 2017 08:19:09 -0000


    [ https://issues.apache.org/jira/browse/FLINK-3089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16130083#comment-16130083 ] 

Stefan Richter commented on FLINK-3089:
---------------------------------------

[~sihuazhou] one thing that we would also need to discuss is what this means for checkpoints/savepoints when operating in event time. How do you want to solve the problem that results are no longer deterministic after recovery/resume? The current workaround (using the timer service) has clearly more overhead, but does not suffer from this problem.

> State API Should Support Data Expiration (State TTL)
> ----------------------------------------------------
>
>                 Key: FLINK-3089
>                 URL: https://issues.apache.org/jira/browse/FLINK-3089
>             Project: Flink
>          Issue Type: New Feature
>          Components: DataStream API, State Backends, Checkpointing
>            Reporter: Niels Basjes
>
> In some usecases (webanalytics) there is a need to have a state per visitor on a website (i.e. keyBy(sessionid) ).
> At some point the visitor simply leaves and no longer creates new events (so a special 'end of session' event will not occur).
> The only way to determine that a visitor has left is by choosing a timeout, like "After 30 minutes no events we consider the visitor 'gone'".
> Only after this (chosen) timeout has expired should we discard this state.
> In the Trigger part of Windows we can set a timer and close/discard this kind of information. But that introduces the buffering effect of the window (which in some scenarios is unwanted).
> What I would like is to be able to set a timeout on a specific state which I can update afterwards.
> This makes it possible to create a map function that assigns the right value and that discards the state automatically.


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)