flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fabian Hueske (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-1683) Scheduling preferences for non-unary tasks are not correctly computed
Date Tue, 10 Mar 2015 20:03:38 GMT
Fabian Hueske created FLINK-1683:

             Summary: Scheduling preferences for non-unary tasks are not correctly computed
                 Key: FLINK-1683
                 URL: https://issues.apache.org/jira/browse/FLINK-1683
             Project: Flink
          Issue Type: Bug
          Components: JobManager
    Affects Versions: 0.8.1, 0.9
            Reporter: Fabian Hueske
            Assignee: Fabian Hueske
             Fix For: 0.9, 0.8.2

When computing scheduling preferences for an execution task, the JobManager looks at the assigned
instances of all its input execution tasks and returns a preference only if not more than
8 instances have been found (if the input of a tasks is distributed across more than 8 tasks,
local scheduling won't help a lot in any case).

However, the JobManager treats all input execution tasks the same and does not distinguish
between different logical input. The effect is that a join tasks with one broadcasted and
one locally forwarded task is not locally assigned towards its locally forwarded input.

This can have a significant impact on the performance of tasks that have more than one input
and which rely on local forwarding and co-located task scheduling.

This message was sent by Atlassian JIRA

View raw message