hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sunil G (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4479) Retrospect app-priority in pendingOrderingPolicy during recovering applications
Date Mon, 21 Dec 2015 12:51:46 GMT

    [ https://issues.apache.org/jira/browse/YARN-4479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15066415#comment-15066415

Sunil G commented on YARN-4479:

Hi [~Naganarasimha]
bq.IUC Sunil G also wanted to say the same ?
I meant in slight different way. With existing approach, any application which was running
earlier to RM restart will also be in RUNNING state. It may starve as per the scenario which
you and Rohith mentioned, but the state of application will be running.

Also adding to the existing discussion, I would like to point out few things.
This patch tries to activate all applications which were running before RM restart happened.
It may be getting activated with different order, but it is trying to put all these apps in
the activated list of scheduler (App state will be RUNNING still). 
1. After restart with or without ordering, only highest priority app will be selected for
scheduling from activated list. This same behavior is happening before RM restart also. SO
there seems no impact with this. Pls correct me if I am wrong.
2. All containers which were running earlier will still continue, and all pending requests
will be updated/refreshed and this is from ApplicationMasterService thread. So if all earlier
running apps are activated,then behavior will be same from scheduler end, correct?

Being said all this points, I also feel that we may need to add more complex code to keep
the same order as you proposed. So if there are no major impacts, I think the approach taken
in this patch looks fine. Thoughts?

> Retrospect app-priority in pendingOrderingPolicy during recovering applications
> -------------------------------------------------------------------------------
>                 Key: YARN-4479
>                 URL: https://issues.apache.org/jira/browse/YARN-4479
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: api, resourcemanager
>            Reporter: Rohith Sharma K S
>            Assignee: Rohith Sharma K S
>         Attachments: 0001-YARN-4479.patch
> Currently, same ordering policy is used for pending applications and active applications.
When priority is configured for an applications, during recovery high priority application
get activated first. It is possible that low priority job was submitted and running state.

> This causes low priority job in starvation after recovery

This message was sent by Atlassian JIRA

View raw message