flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Maximilian Michels (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (FLINK-2317) Stream Jobs with Windows Leave Lingering State
Date Mon, 05 Oct 2015 10:10:27 GMT

     [ https://issues.apache.org/jira/browse/FLINK-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Maximilian Michels resolved FLINK-2317.
       Resolution: Fixed
    Fix Version/s: 0.10

I'm assuming this has been fixed while redoing the window mechanism. If not, please reopen.

> Stream Jobs with Windows Leave Lingering State
> ----------------------------------------------
>                 Key: FLINK-2317
>                 URL: https://issues.apache.org/jira/browse/FLINK-2317
>             Project: Flink
>          Issue Type: Bug
>          Components: Streaming
>    Affects Versions: 0.9, 0.9.1
>            Reporter: Aljoscha Krettek
>            Priority: Blocker
>             Fix For: 0.10
> The problem is that some components in the windowing system create Threads that are not
properly cleaned up after a Job finished (is canceled). Specifically these are the {{CentralCheck}}
thread of the {{GroupedActiveDiscretizer}} and the {{TimeCheck}} thread in {{TimeTriggerPolicy}}.
These threads have references to other parts of the windowing system, such as the window buffers.
This then means that the complete state of the buffers at the time a job is canceled stays
around indefinitely. Over time this will fill up the memory and make successive windowing
jobs incredibly slow or lead to GC overheat exceptions.
> Also, to prevent stuff like this from happening in the future we should put all streaming
Tests on a common {{TestBase}} that verifies that we have no lingering threads after job execution.

This message was sent by Atlassian JIRA

View raw message