hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sunil G (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-2004) Priority scheduling support in Capacity scheduler
Date Tue, 30 Jun 2015 14:36:06 GMT

     [ https://issues.apache.org/jira/browse/YARN-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sunil G updated YARN-2004:
--------------------------
    Attachment: 0009-YARN-2004.patch

Thank you very much [~jianhe] and [~leftnoteasy] for the review and comments

- bq.The app priority seems only used for pending applications, how about priority support
for the actively running applications ?
As mentioned in Wangda's comments, I was using application priority comparision in {{SchedulerApplicationAttempt#compareInputOrderTo}}.
However its better to do the same in {{FifoComparator}}, and I have updated SchedulableEntity
interface to get priroity for same.

- bq.Should we do priority comparison first (treat priority as "class") OR combination of
them (treat priority as "factor").
I feel we can do the priority comparision first. Do you see any specific usecase for priority
as "factor"

- bq.We may just move the check into RMAppManager.
Adding further to Wangda's points, We had a PriorityManager earlier to store the same. Since
we associated the priroity closer to Scehduler, it is now handled here. 

- bq.updateApplicationPriority
In this patch I have handled this scenario in RMAppImpl (By handling a new event from Scheduler
and storing in StateStore) and CS. RMAppImpl changes are updated in YARN-2003. Please suggest
whether I need to handle this specific scenario in a separate ticket (RMAppImpl and CS together).

I have fixed all the other comments as mentioned. Kindly help to check.

> Priority scheduling support in Capacity scheduler
> -------------------------------------------------
>
>                 Key: YARN-2004
>                 URL: https://issues.apache.org/jira/browse/YARN-2004
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacityscheduler
>            Reporter: Sunil G
>            Assignee: Sunil G
>         Attachments: 0001-YARN-2004.patch, 0002-YARN-2004.patch, 0003-YARN-2004.patch,
0004-YARN-2004.patch, 0005-YARN-2004.patch, 0006-YARN-2004.patch, 0007-YARN-2004.patch, 0008-YARN-2004.patch,
0009-YARN-2004.patch
>
>
> Based on the priority of the application, Capacity Scheduler should be able to give preference
to application while doing scheduling.
> Comparator<FiCaSchedulerApp> applicationComparator can be changed as below.		
> 1.	Check for Application priority. If priority is available, then return the highest
priority job.
> 2.	Otherwise continue with existing logic such as App ID comparison and then TimeStamp
comparison.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message