Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 7D9E0200C6E for ; Mon, 8 May 2017 18:41:10 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 7C3AF160BA5; Mon, 8 May 2017 16:41:10 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id C4013160BA2 for ; Mon, 8 May 2017 18:41:09 +0200 (CEST) Received: (qmail 18090 invoked by uid 500); 8 May 2017 16:41:07 -0000 Mailing-List: contact dev-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@flink.apache.org Delivered-To: mailing list dev@flink.apache.org Received: (qmail 18066 invoked by uid 99); 8 May 2017 16:41:07 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 08 May 2017 16:41:07 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 1AE09C0E9B for ; Mon, 8 May 2017 16:41:07 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -100.002 X-Spam-Level: X-Spam-Status: No, score=-100.002 tagged_above=-999 required=6.31 tests=[RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id ruH-GPH5MgHt for ; Mon, 8 May 2017 16:41:05 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id E255B5F477 for ; Mon, 8 May 2017 16:41:04 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 74851E0641 for ; Mon, 8 May 2017 16:41:04 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 3211321DF6 for ; Mon, 8 May 2017 16:41:04 +0000 (UTC) Date: Mon, 8 May 2017 16:41:04 +0000 (UTC) From: "Fabian Hueske (JIRA)" To: dev@flink.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (FLINK-6491) Add QueryConfig to specify state retention time for streaming queries MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Mon, 08 May 2017 16:41:10 -0000 Fabian Hueske created FLINK-6491: ------------------------------------ Summary: Add QueryConfig to specify state retention time for streaming queries Key: FLINK-6491 URL: https://issues.apache.org/jira/browse/FLINK-6491 Project: Flink Issue Type: Bug Components: Table API & SQL Affects Versions: 1.3.0 Reporter: Fabian Hueske Assignee: sunjincheng Priority: Critical By now we have a couple of streaming operators (group-windows, over-windows, non-windowed aggregations) that require operator state. Since state is not automatically cleaned-up by Flink, we need to add a mechanism to configure a state retention time. If configured, a query will retain state for a specified period of state inactivity. If state is not accessed within this period of time, it will be cleared. I propose to add two parameters for this, a min and a max retention time. The min retention time specifies the earliest time and the max retention time the latest time when state is cleared. The reasoning for having two parameters is that we can avoid to register many timers if we have more freedom when to discard state. This issue also introduces a QueryConfig object which can be passed to a streaming query, when it is emitted to a TableSink or converted to a DataStream (append or retraction). -- This message was sent by Atlassian JIRA (v6.3.15#6346)