flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stephan Ewen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-1094) Simplify input split assignments
Date Tue, 09 Sep 2014 16:50:28 GMT

    [ https://issues.apache.org/jira/browse/FLINK-1094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14127211#comment-14127211

Stephan Ewen commented on FLINK-1094:

I would like to solve this as part of improving the JobManager data structures.

> Simplify input split assignments
> --------------------------------
>                 Key: FLINK-1094
>                 URL: https://issues.apache.org/jira/browse/FLINK-1094
>             Project: Flink
>          Issue Type: Improvement
>          Components: JobManager
>            Reporter: Stephan Ewen
>            Assignee: Stephan Ewen
>            Priority: Minor
> The input split assigner is currently one shared instances across jobs and inputs. The
mapping between splits, split type, and assigner is very inflexible, as it requires changing
an internal class in the runtime project.
> The following changes will make it simpler and more efficient:
>  - Attach the split assigner to the job vertex (the ExecutionJobVertex) for the task
that consumes the input splits.
>  - Move the input split assigner interfaces into the core project, as well as the simple
base implementations of the assigners (default and locality aware)
>  - Let input split producers (such as input formats) create their own assigners.

This message was sent by Atlassian JIRA

View raw message