hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2003) Support to process Job priority from Submission Context in AppAttemptAddedSchedulerEvent [RM side]
Date Tue, 17 Mar 2015 23:08:39 GMT

    [ https://issues.apache.org/jira/browse/YARN-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14366273#comment-14366273

Wangda Tan commented on YARN-2003:

Hi [~sunilg],
Thanks for working on this. I took a quick look at YARN-2003 / YARN-2004. Some overall comments.

First I want to describe my thinkings of how RM side workflow look like:
- When application submit to RMAppManager, it will simply pass priority set by user to RMApp
(see (1))
- RMApp will finally create APP_ATTEMPT_ADDED, queue itself will normalize priority (reject
it / convert it from label to number, etc. And set new priority to RMApp), set priority to
- Scheduler uses priority in SchedulerApplicationAttempt and Queue to make scheduling decisions
- If user ask for changing application priority, or admin changes priority configuration,
event may need send to scheduler to update inner applications/queues.
- When user requires priority of application via CLI/Web-UI, ApplicationPriorityManager will
convert number to label (if possible) and show to user.

(1). We don't have to add too much logic here, and if we can simply handle it inside scheduler,
when configuration changed (like label->integer priority mapping changed), it can be handled
by scheduler itself.

Back to your patches, several major differences are:
- RMAppManager takes responsibility to check ACL for priority, which I think not proper (ACL
is always managed by scheduler queues) and not correct (when queue configuration changed,
you cannot do rechecking application priority if new app attempt created.).
- As same as above, queues should check ACLs for priority.
- You may not need add priority to SchedulerApplication, it seems not necessary to me.

> Support to process Job priority from Submission Context in AppAttemptAddedSchedulerEvent
[RM side]
> --------------------------------------------------------------------------------------------------
>                 Key: YARN-2003
>                 URL: https://issues.apache.org/jira/browse/YARN-2003
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Sunil G
>            Assignee: Sunil G
>         Attachments: 0001-YARN-2003.patch, 0002-YARN-2003.patch, 0003-YARN-2003.patch,
> AppAttemptAddedSchedulerEvent should be able to receive the Job Priority from Submission
Context and store.
> Later this can be used by Scheduler.

This message was sent by Atlassian JIRA

View raw message