hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jian He (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2004) Priority scheduling support in Capacity scheduler
Date Tue, 30 Jun 2015 00:06:06 GMT

    [ https://issues.apache.org/jira/browse/YARN-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606697#comment-14606697
] 

Jian He commented on YARN-2004:
-------------------------------

thanks Sunil ! some comments on the patch:


- The app priority seems only used for pending applications, how about priority support for
the actively running applications ?
- “default_application_priority”: the convention is to use “-” instead of “_”;
similarly, the max-application-priority.
- this should not happen
{code}
    if (null == queue) {
      throw new YarnException(
          "During application init/update, failure occured due to an unknown "
              + " queue name '" + queueName + "' from priority authentication");
    }
{code}
because queue will never be null, see below code in ClientRMService
{code}
    if (submissionContext.getQueue() == null) {
      submissionContext.setQueue(YarnConfiguration.DEFAULT_QUEUE_NAME);
    }
{code}

- max-application-priority is defined in yarn-site.xml, but here it’s retrieved from capacity-scheduler.xml.
We may just move the check into RMAppManager.
{code}
 if (priority.getPriority() > getMaxClusterLevelAppPriority()) {
      throw new YarnException("Invalid priority as Queue: " + queueName
          + " cannot support more than priority '"
          + getMaxClusterLevelAppPriority() + "'");
    }
{code}
- updateApplicationPriority - I think we don’t need to add an unused API now. we can do
this later when implementing the functionality of updating app priority



> Priority scheduling support in Capacity scheduler
> -------------------------------------------------
>
>                 Key: YARN-2004
>                 URL: https://issues.apache.org/jira/browse/YARN-2004
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacityscheduler
>            Reporter: Sunil G
>            Assignee: Sunil G
>         Attachments: 0001-YARN-2004.patch, 0002-YARN-2004.patch, 0003-YARN-2004.patch,
0004-YARN-2004.patch, 0005-YARN-2004.patch, 0006-YARN-2004.patch, 0007-YARN-2004.patch, 0008-YARN-2004.patch
>
>
> Based on the priority of the application, Capacity Scheduler should be able to give preference
to application while doing scheduling.
> Comparator<FiCaSchedulerApp> applicationComparator can be changed as below.		
> 1.	Check for Application priority. If priority is available, then return the highest
priority job.
> 2.	Otherwise continue with existing logic such as App ID comparison and then TimeStamp
comparison.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message