hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sunil G (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (YARN-5802) Application priority updates add pending apps to running ordering policy
Date Sun, 30 Oct 2016 04:59:58 GMT

    [ https://issues.apache.org/jira/browse/YARN-5802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15619223#comment-15619223
] 

Sunil G edited comment on YARN-5802 at 10/30/16 4:59 AM:
---------------------------------------------------------

Good catch. After app priority, there were few optimizations done to orderingPolicies. And
hence two policies are now used. Priority update should consider both.

App priority can be updated when app is in ACCEPTED or RUNNING state. Hence the scheduler
can be either in {{pendingOrderingPolicy}} or in normal {{orderingPolicy}}. So please check
whether app is present in {{pendingOrderingPolicy}} or {{orderingPolicy}}, the update back
to same object. Current approach in the patch is not  handling runtime update priority for
accepted/pending apps.

We might need to add {{contains}} method to OrderingPolicy interface for this.


was (Author: sunilg):
Good catch. After app priority, there were few optimizations done to orderingPolicies. And
hence two policies are now used. Priority update should consider both.

App priority can be update when app is in ACCEPTED or RUNNING state. Hence the scheduler can
be in either in {{pendingOrderingPolicy}} or in normal {{orderingPolicy}}. So please check
whether app is present in {{pendingOrderingPolicy}} or {{orderingPolicy}}, the update back
to same object. Current approach in the patch is not  handling runtime update priority for
accepted/pending apps.

> Application priority updates add pending apps to running ordering policy
> ------------------------------------------------------------------------
>
>                 Key: YARN-5802
>                 URL: https://issues.apache.org/jira/browse/YARN-5802
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Bibin A Chundatt
>            Assignee: Bibin A Chundatt
>            Priority: Critical
>         Attachments: YARN-5802.0001.patch
>
>
> {{LeafQueue#updateApplicationPriority}}
> {code}
>  getOrderingPolicy().removeSchedulableEntity(attempt);
>       // Update new priority in SchedulerApplication
>       attempt.setPriority(newAppPriority);
>       getOrderingPolicy().addSchedulableEntity(attempt);
> {code}
> We should add again to ordering policy only when  attempt available in first case.Else
during application attempt removal will try to iterate on killed application still available
in pending Ordering policy.Which can cause RM to crash.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message