Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Sun, 30 Oct 2016 04:59:58 +0000 (UTC)
From: "Sunil G (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.13016377.1477772236000.127024.1477803598503@Atlassian.JIRA>
In-Reply-To: <JIRA.13016377.1477772236000@Atlassian.JIRA>
References: <JIRA.13016377.1477772236000@Atlassian.JIRA> <JIRA.13016377.1477772236909@arcas>
Subject: [jira] [Comment Edited] (YARN-5802) Application priority updates
 add pending apps to running ordering policy
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Sun, 30 Oct 2016 05:00:00 -0000


    [ https://issues.apache.org/jira/browse/YARN-5802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15619223#comment-15619223 ] 

Sunil G edited comment on YARN-5802 at 10/30/16 4:59 AM:
---------------------------------------------------------

Good catch. After app priority, there were few optimizations done to orderingPolicies. And hence two policies are now used. Priority update should consider both.

App priority can be updated when app is in ACCEPTED or RUNNING state. Hence the scheduler can be either in {{pendingOrderingPolicy}} or in normal {{orderingPolicy}}. So please check whether app is present in {{pendingOrderingPolicy}} or {{orderingPolicy}}, the update back to same object. Current approach in the patch is not  handling runtime update priority for accepted/pending apps.

We might need to add {{contains}} method to OrderingPolicy interface for this.


was (Author: sunilg):
Good catch. After app priority, there were few optimizations done to orderingPolicies. And hence two policies are now used. Priority update should consider both.

App priority can be update when app is in ACCEPTED or RUNNING state. Hence the scheduler can be in either in {{pendingOrderingPolicy}} or in normal {{orderingPolicy}}. So please check whether app is present in {{pendingOrderingPolicy}} or {{orderingPolicy}}, the update back to same object. Current approach in the patch is not  handling runtime update priority for accepted/pending apps.

> Application priority updates add pending apps to running ordering policy
> ------------------------------------------------------------------------
>
>                 Key: YARN-5802
>                 URL: https://issues.apache.org/jira/browse/YARN-5802
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Bibin A Chundatt
>            Assignee: Bibin A Chundatt
>            Priority: Critical
>         Attachments: YARN-5802.0001.patch
>
>
> {{LeafQueue#updateApplicationPriority}}
> {code}
>  getOrderingPolicy().removeSchedulableEntity(attempt);
>       // Update new priority in SchedulerApplication
>       attempt.setPriority(newAppPriority);
>       getOrderingPolicy().addSchedulableEntity(attempt);
> {code}
> We should add again to ordering policy only when  attempt available in first case.Else during application attempt removal will try to iterate on killed application still available in pending Ordering policy.Which can cause RM to crash.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org