Mailing-List: contact issues-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@flink.apache.org
Date: Wed, 11 Oct 2017 12:40:00 +0000 (UTC)
From: "Gerard Garcia (JIRA)" <jira@apache.org>
To: issues@flink.apache.org
Message-ID: <JIRA.13105762.1506607881000.27613.1507725600302@Atlassian.JIRA>
In-Reply-To: <JIRA.13105762.1506607881000@Atlassian.JIRA>
References: <JIRA.13105762.1506607881000@Atlassian.JIRA> <JIRA.13105762.1506607881741@jira-lw-us.apache.org>
Subject: [jira] [Commented] (FLINK-7731) Trigger on GlobalWindow does not
 clean state completely
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Wed, 11 Oct 2017 12:40:05 -0000


    [ https://issues.apache.org/jira/browse/FLINK-7731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16200208#comment-16200208 ] 

Gerard Garcia commented on FLINK-7731:
--------------------------------------

I tried that but it still doesn't work. I have been doing some debugging and it seems to me that the timer never gets triggered. It may be that what is happening is that the {{onProcessingTime}} callback is only triggered when there are contents in the window state (all code snippets are from version 1.3.2):

org.apache.flink.streaming.runtime.operators.windowing.WindowOperator#onProcessingTime()
{code:java}
		ACC contents = null;
		if (windowState != null) {
			contents = windowState.get();
		}

		if (contents != null) {
			TriggerResult triggerResult = triggerContext.onProcessingTime(timer.getTimestamp());
			if (triggerResult.isFire()) {
				emitWindowContents(triggerContext.window, contents);
			}
			if (triggerResult.isPurge()) {
				windowState.clear();
			}
		}
{code}

and that state is being cleared when the trigger result is type purge:

org.apache.flink.streaming.runtime.operators.windowing.WindowOperator#processElement()
{code:java}
				TriggerResult triggerResult = triggerContext.onElement(element);

				if (triggerResult.isFire()) {
					ACC contents = windowState.get();
					if (contents == null) {
						continue;
					}
					emitWindowContents(window, contents);
				}

				if (triggerResult.isPurge()) {
					windowState.clear();
				}
{code}


> Trigger on GlobalWindow does not clean state completely
> -------------------------------------------------------
>
>                 Key: FLINK-7731
>                 URL: https://issues.apache.org/jira/browse/FLINK-7731
>             Project: Flink
>          Issue Type: Bug
>          Components: Core, DataStream API
>    Affects Versions: 1.3.2
>            Reporter: Gerard Garcia
>            Priority: Minor
>
> I have an operator that consists of:
> CoGroup Datastream -> GlobalWindow -> CustomTrigger -> Apply function
> The custom trigger fires and purges the elements after it has received the expected number of elements (or when a timeout fires) from one of the streams and the apply function merges the elements with the ones received from the other stream. It appears that the state of the operator grows continuously so it seems it never gets completely cleaned.
> There is a discussion in http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Clean-GlobalWidnow-state-td15613.html that suggests that it may be a bug.
> This job reproduces the issue: https://github.com/GerardGarcia/flink-global-window-growing-state


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)