Return-Path: X-Original-To: apmail-hadoop-yarn-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-yarn-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id E442A10858 for ; Wed, 11 Jun 2014 18:30:07 +0000 (UTC) Received: (qmail 8297 invoked by uid 500); 11 Jun 2014 18:30:06 -0000 Delivered-To: apmail-hadoop-yarn-issues-archive@hadoop.apache.org Received: (qmail 7631 invoked by uid 500); 11 Jun 2014 18:30:05 -0000 Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: yarn-issues@hadoop.apache.org Delivered-To: mailing list yarn-issues@hadoop.apache.org Received: (qmail 7621 invoked by uid 99); 11 Jun 2014 18:30:05 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 11 Jun 2014 18:30:05 +0000 Date: Wed, 11 Jun 2014 18:30:05 +0000 (UTC) From: "Mayank Bansal (JIRA)" To: yarn-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (YARN-2022) Preempting an Application Master container can be kept as least priority when multiple applications are marked for preemption by ProportionalCapacityPreemptionPolicy MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/YARN-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14028174#comment-14028174 ] Mayank Bansal commented on YARN-2022: ------------------------------------- hi [~sunilg] Thanks for the patch. There is small addition we need to do in the approach which is as follows we need to consider below parameters into account yarn.scheduler.capacity.maximum-am-resource-percent / yarn.scheduler.capacity..maximum-am-resource-percent It would be if user have set yarn.scheduler.capacity..maximum-am-resource-percent in the queue then we can not preempt AM even if we didn't reach to full resource need from the queue. If user didn't set that queue level setting then we need to check if we are not avoiding yarn.scheduler.capacity.maximum-am-resource-percent constraint as well if these two constraints are not avoided and still we have some ams which we need to kill then yes we can go with the approach you put in your patch. Thanks, Mayank > Preempting an Application Master container can be kept as least priority when multiple applications are marked for preemption by ProportionalCapacityPreemptionPolicy > --------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > Key: YARN-2022 > URL: https://issues.apache.org/jira/browse/YARN-2022 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager > Affects Versions: 2.4.0 > Reporter: Sunil G > Assignee: Sunil G > Attachments: YARN-2022-DesignDraft.docx, YARN-2022.2.patch, YARN-2022.3.patch, Yarn-2022.1.patch > > > Cluster Size = 16GB [2NM's] > Queue A Capacity = 50% > Queue B Capacity = 50% > Consider there are 3 applications running in Queue A which has taken the full cluster capacity. > J1 = 2GB AM + 1GB * 4 Maps > J2 = 2GB AM + 1GB * 4 Maps > J3 = 2GB AM + 1GB * 2 Maps > Another Job J4 is submitted in Queue B [J4 needs a 2GB AM + 1GB * 2 Maps ]. > Currently in this scenario, Jobs J3 will get killed including its AM. > It is better if AM can be given least priority among multiple applications. In this same scenario, map tasks from J3 and J2 can be preempted. > Later when cluster is free, maps can be allocated to these Jobs. -- This message was sent by Atlassian JIRA (v6.2#6252)