hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2004) Priority scheduling support in Capacity scheduler
Date Tue, 30 Jun 2015 18:51:06 GMT

    [ https://issues.apache.org/jira/browse/YARN-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14608857#comment-14608857
] 

Wangda Tan commented on YARN-2004:
----------------------------------

Thanks for update, [~sunilg], comments to latest patch:

1)
bq. I feel we can do the priority comparision first. Do you see any specific usecase for priority
as "factor"
Fair scheduler currently uses it as "factor", see {{FairScheduler#getAppWeight}}

2)
SchedulerApplicationAttempt/SchedulerApplication.appPriority should be volatile. I found there're
some other fields need to be changed, not caused by your patch. For example: SchedulerApplication.currentAttempt,
etc. I suggest we make appPriority correct in this patch, and address others in separated
ticket.

3)
Not caused by your patch, applicationComparator should be removed, and pendingApps in LeafQueue
should use FifoOrderingPolicy to compare, we can do this in separated patch.

4)
dfltAppPriorityPerQueue should be default ..

5)
Is this check nececessary in SchedulerApplication.setPriority:
{code}
78	    if (null != currentAttempt) {
79	      currentAttempt.setApplicationPriority(priority);
80	    }
81	  }
{code}
Should we simply prohibit changing application priority when app's in submitting stage?

6)
Tests:
- Add a test to verify updateApplicationPriority works?
- Add end-to-end test to verify application priority works? (Not only check q.getApplications().iterator..)

> Priority scheduling support in Capacity scheduler
> -------------------------------------------------
>
>                 Key: YARN-2004
>                 URL: https://issues.apache.org/jira/browse/YARN-2004
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacityscheduler
>            Reporter: Sunil G
>            Assignee: Sunil G
>         Attachments: 0001-YARN-2004.patch, 0002-YARN-2004.patch, 0003-YARN-2004.patch,
0004-YARN-2004.patch, 0005-YARN-2004.patch, 0006-YARN-2004.patch, 0007-YARN-2004.patch, 0008-YARN-2004.patch,
0009-YARN-2004.patch
>
>
> Based on the priority of the application, Capacity Scheduler should be able to give preference
to application while doing scheduling.
> Comparator<FiCaSchedulerApp> applicationComparator can be changed as below.		
> 1.	Check for Application priority. If priority is available, then return the highest
priority job.
> 2.	Otherwise continue with existing logic such as App ID comparison and then TimeStamp
comparison.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message