hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6443) Allow for Priority order relaxing in favor of improved node/rack locality
Date Wed, 05 Apr 2017 05:29:42 GMT

    [ https://issues.apache.org/jira/browse/YARN-6443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15956325#comment-15956325
] 

Wangda Tan commented on YARN-6443:
----------------------------------

[~arun.suresh@gmail.com],

I was thinking about similar things while working on YARN-5139. One issue I can think of is,
it will break ordering of applications/queues/schedulerKeys.

The ideal solution in my mind is pass multiple nodes while doing scheduling and scoring/sorting
nodes based on placement policy (for example, based on locality).

There're some optimizations we can make to reduce the overhead of sorting. I put some of the
stuffs in early patch of YARN-5139: https://issues.apache.org/jira/secure/attachment/12825346/wip-4.YARN-5139.patch#file-36
please check LocalityNodesScorer.java. 

I think we can use the similar approach to optimize affinity/anti-affinity which I mentioned
offline.

Please let me know your thoughts.

> Allow for Priority order relaxing in favor of improved node/rack locality 
> --------------------------------------------------------------------------
>
>                 Key: YARN-6443
>                 URL: https://issues.apache.org/jira/browse/YARN-6443
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: capacity scheduler, fairscheduler
>            Reporter: Arun Suresh
>            Assignee: Arun Suresh
>
> Currently the Schedulers examine an applications pending Requests in Priority order.
This JIRA proposes to introduce a flag (either via the ApplicationMasterService::registerApplication()
or via some Scheduler configuration) to favor an ordering that is baised to the node that
is currently heartbeating by relaxing the priority constraint.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message