flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fabian Hueske (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-1445) Add support to enforce local input split assignment
Date Sat, 24 Jan 2015 22:30:34 GMT
Fabian Hueske created FLINK-1445:

             Summary: Add support to enforce local input split assignment
                 Key: FLINK-1445
                 URL: https://issues.apache.org/jira/browse/FLINK-1445
             Project: Flink
          Issue Type: New Feature
          Components: Java API, JobManager
    Affects Versions: 0.9
            Reporter: Fabian Hueske
            Priority: Minor

In some scenarios data sources cannot remotely read data as for example in distributed cluster
setups where each machine stores data on its local FS which cannot be remotely read.

In order to enable such use cases with Flink, we need to 
1) add support for enforcing local input split reading.
2) ensure that each input split can be locally read by at least one data source task which
means to influence the scheduling of data source tasks. 

This message was sent by Atlassian JIRA

View raw message