Mailing-List: contact issues-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@flink.apache.org
Date: Fri, 22 May 2015 15:23:17 +0000 (UTC)
From: "Stephan Ewen (JIRA)" <jira@apache.org>
To: issues@flink.apache.org
Message-ID: <JIRA.12832184.1432308150000.10797.1432308197408@Atlassian.JIRA>
In-Reply-To: <JIRA.12832184.1432308150000@Atlassian.JIRA>
References: <JIRA.12832184.1432308150000@Atlassian.JIRA>
 <JIRA.12832184.1432308150770@arcas>
Subject: [jira] [Created] (FLINK-2084) Create a dedicated streaming mode
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

Stephan Ewen created FLINK-2084:
-----------------------------------

             Summary: Create a dedicated streaming mode
                 Key: FLINK-2084
                 URL: https://issues.apache.org/jira/browse/FLINK-2084
             Project: Flink
          Issue Type: Improvement
          Components: Distributed Runtime
    Affects Versions: 0.9
            Reporter: Stephan Ewen
            Assignee: Stephan Ewen
             Fix For: 0.9


As per discussion on the mailing list

http://mail-archives.apache.org/mod_mbox/flink-dev/201505.mbox/browser

 - We add a dedicated streaming mode for now. The streaming mode supersedes the batch mode, so it can run both type of programs.

 - The streaming mode sets the memory manager to "lazy allocation".
    -> So long as it runs pure streaming jobs, the full heap will be
available to window buffers and UDFs.
    -> Batch programs can still run, so mixed workloads are not prevented.
Batch programs are a bit less robust there, because the memory manager does not pre-allocate memory. UDFs can eat into Flink's memory portion.

 - The streaming mode starts the necessary configured components/services for state backups


Over the next versions, we want to bring these things together:
  - use the managed memory for window buffers
  - on-demand starting of the state backend

Then, we deprecate the streaming mode, let both modes start the cluster in the same way.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)