spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-25780) Scheduling the tasks which have no higher level locality first
Date Fri, 19 Oct 2018 07:11:00 GMT

    [ https://issues.apache.org/jira/browse/SPARK-25780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16656355#comment-16656355
] 

Apache Spark commented on SPARK-25780:
--------------------------------------

User '10110346' has created a pull request for this issue:
https://github.com/apache/spark/pull/22774

> Scheduling the tasks which have no higher level locality first
> --------------------------------------------------------------
>
>                 Key: SPARK-25780
>                 URL: https://issues.apache.org/jira/browse/SPARK-25780
>             Project: Spark
>          Issue Type: Improvement
>          Components: Scheduler
>    Affects Versions: 3.0.0
>            Reporter: liuxian
>            Priority: Minor
>
> For example:
> An application has two executors: (exe1, host1), (exe2,host2).
> And 3 tasks with locality: \{task0, Seq(TaskLocation("host1", "exec1"))}, \{task1, Seq(TaskLocation("host1",
"exec1"), TaskLocation("host2"))},  \{task2, Seq(TaskLocation("host2")}
> If task0 is runing in exe1, when `allowedLocality` is NODE_LOCAL for exe2, it is better
to schedule task2 fisrt, not task1, because task1 may be scheduled to exe1 later.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message