Mailing-List: contact dev-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@flink.apache.org
Date: Tue, 12 May 2015 19:57:01 +0000 (UTC)
From: "Stephan Ewen (JIRA)" <jira@apache.org>
To: dev@flink.apache.org
Message-ID: <JIRA.12829310.1431460589000.92503.1431460621655@Atlassian.JIRA>
In-Reply-To: <JIRA.12829310.1431460589000@Atlassian.JIRA>
References: <JIRA.12829310.1431460589000@Atlassian.JIRA>
 <JIRA.12829310.1431460589942@arcas>
Subject: [jira] [Created] (FLINK-2004) Memory leack in presence of failed
 checkpoints in KafkaSource
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

Stephan Ewen created FLINK-2004:
-----------------------------------

             Summary: Memory leack in presence of failed checkpoints in KafkaSource
                 Key: FLINK-2004
                 URL: https://issues.apache.org/jira/browse/FLINK-2004
             Project: Flink
          Issue Type: Bug
          Components: Streaming
    Affects Versions: 0.9
            Reporter: Stephan Ewen
            Assignee: Robert Metzger
            Priority: Critical
             Fix For: 0.9


Checkpoints that fail never send a commit message to the tasks.

Maintaining a map of all pending checkpoints introduces a memory leak, as entries for failed checkpoints will never be removed.

Approaches to fix this:
  - The source cleans up entries from older checkpoints once a checkpoint is committed (simple implementation in a linked hash map)
  - The commit message could include the optional state handle (source needs not maintain the map)
  - The checkpoint coordinator could send messages for failed checkpoints?


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)