hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun Suresh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6443) Allow for Priority order relaxing in favor of improved node/rack locality
Date Fri, 07 Apr 2017 04:15:41 GMT

    [ https://issues.apache.org/jira/browse/YARN-6443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15960249#comment-15960249

Arun Suresh commented on YARN-6443:

Apologies for probably not wording this correctly.

bq. If the app doesn't really care what order two different kinds of requests are given in
as long as they have good locality, why not just submit them at the same priority instead
of having this extra flag?
This is essentially should solve the problem, but currently the allocationReqId is used as
the secondary sorting key to order the SchedulerKeys. What we need is
# Ignore the allocationId is the sort order
# In the event that requests with the same Priority but different allocReqId are outstanding,
we want a way to efficiently examine the key that matches the Node that is HB-ing first before
going over the remaining keys.

The patch you posted seemed to be doing something similar, but I guess that will work if there
are multiple outstanding reqs againsts the same SchedulerKey. What we requires is efficient
matching across schedulerKeys

[~hrsharma], I understand you have a prototype for this - please do post it here.

> Allow for Priority order relaxing in favor of improved node/rack locality 
> --------------------------------------------------------------------------
>                 Key: YARN-6443
>                 URL: https://issues.apache.org/jira/browse/YARN-6443
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: capacity scheduler, fairscheduler
>            Reporter: Arun Suresh
>            Assignee: Hitesh Sharma
> Currently the Schedulers examine an applications pending Requests in Priority order.
This JIRA proposes to introduce a flag (either via the ApplicationMasterService::registerApplication()
or via some Scheduler configuration) to favor an ordering that is baised to the node that
is currently heartbeating by relaxing the priority constraint.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message