hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sunil G (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2004) Priority scheduling support in Capacity scheduler
Date Mon, 20 Apr 2015 16:44:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14503158#comment-14503158

Sunil G commented on YARN-2004:

Thank you very much for the comments.

bq. default of default-priority is -1
I also have similar opinion as told by [~jlowe]. If we are looking for linux like priority
and with range (-N,N), we may need the support of negative. But as a simple comparison, both
do not matter much. For maintainability, I also support use of +ve integer and 0 as default.

bq. We don't need per-user settings to get the basic
A user can submit an application with a given priority.
This priority will be validated against
	1) whether is a valid priority as per the cluster priority list (0:Low, 1:Medium, 2:High)
	2) whether is valid for the given queue config (QueueA {default=Low, max=Medium})
		Hence Low and Medium are accessible for QueueA
	3) ACLs (This will be done with a separate ticket)

Now if user didnt submit app with a priority, we can take the default priority (Here for QueueA
it is Low) configured for given queue.
In earlier patch, this point was not added. I will add the same in subsequent patch.

Coming to the point of discussion, I feel we can do this above design first, and then can
handle per-user priority feature as a separate ticket.
[~leftnoteasy] and [~jlowe] pls suggest your thoughts

bq. There appear to be some missing NULL checks
I am sorry for this, it will be removed.

As suggested, I will change the log part and will upload a new version of patch.

> Priority scheduling support in Capacity scheduler
> -------------------------------------------------
>                 Key: YARN-2004
>                 URL: https://issues.apache.org/jira/browse/YARN-2004
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacityscheduler
>            Reporter: Sunil G
>            Assignee: Sunil G
>         Attachments: 0001-YARN-2004.patch, 0002-YARN-2004.patch, 0003-YARN-2004.patch,
0004-YARN-2004.patch, 0005-YARN-2004.patch
> Based on the priority of the application, Capacity Scheduler should be able to give preference
to application while doing scheduling.
> Comparator<FiCaSchedulerApp> applicationComparator can be changed as below.		
> 1.	Check for Application priority. If priority is available, then return the highest
priority job.
> 2.	Otherwise continue with existing logic such as App ID comparison and then TimeStamp

This message was sent by Atlassian JIRA

View raw message