Return-Path: X-Original-To: apmail-hadoop-yarn-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-yarn-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id EF06518012 for ; Fri, 8 May 2015 21:16:05 +0000 (UTC) Received: (qmail 52935 invoked by uid 500); 8 May 2015 21:16:05 -0000 Delivered-To: apmail-hadoop-yarn-issues-archive@hadoop.apache.org Received: (qmail 52892 invoked by uid 500); 8 May 2015 21:16:05 -0000 Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: yarn-issues@hadoop.apache.org Delivered-To: mailing list yarn-issues@hadoop.apache.org Received: (qmail 52880 invoked by uid 99); 8 May 2015 21:16:05 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 08 May 2015 21:16:05 +0000 Date: Fri, 8 May 2015 21:16:05 +0000 (UTC) From: "Zhijie Shen (JIRA)" To: yarn-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (YARN-2380) The normalizeRequests method in SchedulerUtils always resets the vCore to 1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/YARN-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-2380: ------------------------------ Labels: (was: BB2015-05-TBR) > The normalizeRequests method in SchedulerUtils always resets the vCore to 1 > --------------------------------------------------------------------------- > > Key: YARN-2380 > URL: https://issues.apache.org/jira/browse/YARN-2380 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager > Affects Versions: 2.4.0 > Reporter: Jian Fang > Assignee: Jian Fang > Priority: Critical > Attachments: YARN-2380.patch > > > I added some log info to the method normalizeRequest() as follows. > public static void normalizeRequest( > ResourceRequest ask, > ResourceCalculator resourceCalculator, > Resource clusterResource, > Resource minimumResource, > Resource maximumResource, > Resource incrementResource) { > LOG.info("Before request normalization, the ask capacity: " + ask.getCapability()); > Resource normalized = > Resources.normalize( > resourceCalculator, ask.getCapability(), minimumResource, > maximumResource, incrementResource); > LOG.info("After request normalization, the ask capacity: " + normalized); > ask.setCapability(normalized); > } > The resulted log showed that the vcore in ask was changed from 2 to 1. > 2014-08-01 20:54:15,537 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils (IPC Server handler 4 on 9024): Before request normalization, the ask capacity: > 2014-08-01 20:54:15,537 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils (IPC Server handler 4 on 9024): After request normalization, the ask capacity: > The root cause is the DefaultResourceCalculator calls Resources.createResource(normalizedMemory) to regenerate a new resource with vcore = 1. > This bug is critical and it leads to the mismatch of the request resource and the container resource and many other potential issues if the user requests containers with vcore > 1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)