Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id BB9922009D9 for ; Wed, 18 May 2016 02:13:15 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id BA3871609F5; Wed, 18 May 2016 00:13:15 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id B1B5A160A1F for ; Wed, 18 May 2016 02:13:14 +0200 (CEST) Received: (qmail 91922 invoked by uid 500); 18 May 2016 00:13:13 -0000 Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list yarn-issues@hadoop.apache.org Received: (qmail 91905 invoked by uid 99); 18 May 2016 00:13:13 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 18 May 2016 00:13:13 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 9843B2C1F58 for ; Wed, 18 May 2016 00:13:13 +0000 (UTC) Date: Wed, 18 May 2016 00:13:13 +0000 (UTC) From: "Hadoop QA (JIRA)" To: yarn-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (YARN-4844) Add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Wed, 18 May 2016 00:13:15 -0000 [ https://issues.apache.org/jira/browse/YARN-4844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15287942#comment-15287942 ] Hadoop QA commented on YARN-4844: --------------------------------- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 55 new or modified test files. {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 14s {color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 21s {color} | {color:green} branch-2 passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 26s {color} | {color:green} branch-2 passed with JDK v1.8.0_91 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 16s {color} | {color:green} branch-2 passed with JDK v1.7.0_101 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m 12s {color} | {color:green} branch-2 passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m 25s {color} | {color:green} branch-2 passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m 45s {color} | {color:green} branch-2 passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m 21s {color} | {color:green} branch-2 passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 37s {color} | {color:green} branch-2 passed with JDK v1.8.0_91 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m 25s {color} | {color:green} branch-2 passed with JDK v1.7.0_101 {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 13s {color} | {color:blue} Maven dependency ordering for patch {color} | | {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 27s {color} | {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} | | {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 12s {color} | {color:red} hadoop-sls in the patch failed. {color} | | {color:red}-1{color} | {color:red} compile {color} | {color:red} 3m 4s {color} | {color:red} root in the patch failed with JDK v1.8.0_91. {color} | | {color:red}-1{color} | {color:red} cc {color} | {color:red} 3m 4s {color} | {color:red} root in the patch failed with JDK v1.8.0_91. {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 3m 4s {color} | {color:red} root in the patch failed with JDK v1.8.0_91. {color} | | {color:red}-1{color} | {color:red} compile {color} | {color:red} 3m 43s {color} | {color:red} root in the patch failed with JDK v1.7.0_101. {color} | | {color:red}-1{color} | {color:red} cc {color} | {color:red} 3m 43s {color} | {color:red} root in the patch failed with JDK v1.7.0_101. {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 3m 43s {color} | {color:red} root in the patch failed with JDK v1.7.0_101. {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m 15s {color} | {color:red} root: patch generated 187 new + 4208 unchanged - 113 fixed = 4395 total (was 4321) {color} | | {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m 29s {color} | {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} | | {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m 16s {color} | {color:red} hadoop-sls in the patch failed. {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m 34s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s {color} | {color:red} The patch has 1 line(s) with tabs. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 17s {color} | {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 12s {color} | {color:red} hadoop-sls in the patch failed. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 43s {color} | {color:green} the patch passed with JDK v1.8.0_91 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m 15s {color} | {color:green} the patch passed with JDK v1.7.0_101 {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 21s {color} | {color:green} hadoop-yarn-api in the patch passed with JDK v1.8.0_91. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 4s {color} | {color:green} hadoop-yarn-common in the patch passed with JDK v1.8.0_91. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 21s {color} | {color:green} hadoop-yarn-server-common in the patch passed with JDK v1.8.0_91. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m 18s {color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_91. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 28s {color} | {color:red} hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_91. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 56s {color} | {color:red} hadoop-yarn-client in the patch failed with JDK v1.8.0_91. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 23s {color} | {color:red} hadoop-yarn-applications-distributedshell in the patch failed with JDK v1.8.0_91. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 11s {color} | {color:red} hadoop-sls in the patch failed with JDK v1.8.0_91. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 23s {color} | {color:green} hadoop-yarn-api in the patch passed with JDK v1.7.0_101. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 24s {color} | {color:green} hadoop-yarn-common in the patch passed with JDK v1.7.0_101. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 25s {color} | {color:green} hadoop-yarn-server-common in the patch passed with JDK v1.7.0_101. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 10m 5s {color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_101. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 39s {color} | {color:red} hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_101. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 1m 10s {color} | {color:red} hadoop-yarn-client in the patch failed with JDK v1.7.0_101. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 31s {color} | {color:red} hadoop-yarn-applications-distributedshell in the patch failed with JDK v1.7.0_101. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 17s {color} | {color:red} hadoop-sls in the patch failed with JDK v1.7.0_101. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 20s {color} | {color:green} Patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 104m 17s {color} | {color:black} {color} | \\ \\ || Reason || Tests || | JDK v1.8.0_91 Failed junit tests | hadoop.yarn.client.TestGetGroups | | | hadoop.yarn.client.cli.TestYarnCLI | | | hadoop.yarn.client.api.impl.TestNMClient | | | hadoop.yarn.client.TestRMFailover | | | hadoop.yarn.client.TestResourceTrackerOnHA | | | hadoop.yarn.client.TestApplicationMasterServiceProtocolOnHA | | | hadoop.yarn.client.TestHedgingRequestRMFailoverProxyProvider | | | hadoop.yarn.client.api.impl.TestYarnClient | | | hadoop.yarn.client.api.impl.TestAMRMClient | | | hadoop.yarn.client.api.impl.TestAMRMProxy | | | hadoop.yarn.client.TestApplicationClientProtocolOnHA | | | hadoop.yarn.client.TestResourceManagerAdministrationProtocolPBClientImpl | | | hadoop.yarn.client.api.impl.TestAMRMClientOnRMRestart | | | hadoop.yarn.applications.distributedshell.TestDistributedShell | | | hadoop.yarn.applications.distributedshell.TestDistributedShellWithNodeLabels | | JDK v1.7.0_101 Failed junit tests | hadoop.yarn.client.TestGetGroups | | | hadoop.yarn.client.cli.TestYarnCLI | | | hadoop.yarn.client.api.impl.TestNMClient | | | hadoop.yarn.client.TestRMFailover | | | hadoop.yarn.client.TestResourceTrackerOnHA | | | hadoop.yarn.client.TestApplicationMasterServiceProtocolOnHA | | | hadoop.yarn.client.TestHedgingRequestRMFailoverProxyProvider | | | hadoop.yarn.client.api.impl.TestYarnClient | | | hadoop.yarn.client.api.impl.TestAMRMClient | | | hadoop.yarn.client.api.impl.TestAMRMProxy | | | hadoop.yarn.client.TestApplicationClientProtocolOnHA | | | hadoop.yarn.client.TestResourceManagerAdministrationProtocolPBClientImpl | | | hadoop.yarn.client.api.impl.TestAMRMClientOnRMRestart | | | hadoop.yarn.applications.distributedshell.TestDistributedShell | | | hadoop.yarn.applications.distributedshell.TestDistributedShellWithNodeLabels | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:babe025 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12804529/YARN-4844.8.branch-2.patch | | JIRA Issue | YARN-4844 | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc | | uname | Linux 43594afa2728 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | branch-2 / 264c06a | | Default Java | 1.7.0_101 | | Multi-JDK versions | /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101 | | findbugs | v3.0.0 | | mvninstall | https://builds.apache.org/job/PreCommit-YARN-Build/11514/artifact/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt | | mvninstall | https://builds.apache.org/job/PreCommit-YARN-Build/11514/artifact/patchprocess/patch-mvninstall-hadoop-tools_hadoop-sls.txt | | compile | https://builds.apache.org/job/PreCommit-YARN-Build/11514/artifact/patchprocess/patch-compile-root-jdk1.8.0_91.txt | | cc | https://builds.apache.org/job/PreCommit-YARN-Build/11514/artifact/patchprocess/patch-compile-root-jdk1.8.0_91.txt | | javac | https://builds.apache.org/job/PreCommit-YARN-Build/11514/artifact/patchprocess/patch-compile-root-jdk1.8.0_91.txt | | compile | https://builds.apache.org/job/PreCommit-YARN-Build/11514/artifact/patchprocess/patch-compile-root-jdk1.7.0_101.txt | | cc | https://builds.apache.org/job/PreCommit-YARN-Build/11514/artifact/patchprocess/patch-compile-root-jdk1.7.0_101.txt | | javac | https://builds.apache.org/job/PreCommit-YARN-Build/11514/artifact/patchprocess/patch-compile-root-jdk1.7.0_101.txt | | checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/11514/artifact/patchprocess/diff-checkstyle-root.txt | | mvnsite | https://builds.apache.org/job/PreCommit-YARN-Build/11514/artifact/patchprocess/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt | | mvnsite | https://builds.apache.org/job/PreCommit-YARN-Build/11514/artifact/patchprocess/patch-mvnsite-hadoop-tools_hadoop-sls.txt | | whitespace | https://builds.apache.org/job/PreCommit-YARN-Build/11514/artifact/patchprocess/whitespace-tabs.txt | | findbugs | https://builds.apache.org/job/PreCommit-YARN-Build/11514/artifact/patchprocess/patch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt | | findbugs | https://builds.apache.org/job/PreCommit-YARN-Build/11514/artifact/patchprocess/patch-findbugs-hadoop-tools_hadoop-sls.txt | | unit | https://builds.apache.org/job/PreCommit-YARN-Build/11514/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt | | unit | https://builds.apache.org/job/PreCommit-YARN-Build/11514/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.8.0_91.txt | | unit | https://builds.apache.org/job/PreCommit-YARN-Build/11514/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-applications-distributedshell-jdk1.8.0_91.txt | | unit | https://builds.apache.org/job/PreCommit-YARN-Build/11514/artifact/patchprocess/patch-unit-hadoop-tools_hadoop-sls-jdk1.8.0_91.txt | | unit | https://builds.apache.org/job/PreCommit-YARN-Build/11514/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt | | unit | https://builds.apache.org/job/PreCommit-YARN-Build/11514/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_101.txt | | unit | https://builds.apache.org/job/PreCommit-YARN-Build/11514/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-applications-distributedshell-jdk1.7.0_101.txt | | unit | https://builds.apache.org/job/PreCommit-YARN-Build/11514/artifact/patchprocess/patch-unit-hadoop-tools_hadoop-sls-jdk1.7.0_101.txt | | unit test logs | https://builds.apache.org/job/PreCommit-YARN-Build/11514/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-YARN-Build/11514/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-applications-distributedshell-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-YARN-Build/11514/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_101.txt https://builds.apache.org/job/PreCommit-YARN-Build/11514/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-applications-distributedshell-jdk1.7.0_101.txt | | JDK v1.7.0_101 Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/11514/testReport/ | | modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell hadoop-tools/hadoop-sls U: . | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/11514/console | | Powered by | Apache Yetus 0.2.0 http://yetus.apache.org | This message was automatically generated. > Add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource > -------------------------------------------------------------------- > > Key: YARN-4844 > URL: https://issues.apache.org/jira/browse/YARN-4844 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api > Reporter: Wangda Tan > Assignee: Wangda Tan > Priority: Blocker > Attachments: YARN-4844.1.patch, YARN-4844.2.patch, YARN-4844.3.patch, YARN-4844.4.patch, YARN-4844.5.patch, YARN-4844.6.patch, YARN-4844.7.patch, YARN-4844.8.branch-2.patch, YARN-4844.8.patch > > > We use int32 for memory now, if a cluster has 10k nodes, each node has 210G memory, we will get a negative total cluster memory. > And another case that easier overflows int32 is: we added all pending resources of running apps to cluster's total pending resources. If a problematic app requires too much resources (let's say 1M+ containers, each of them has 3G containers), int32 will be not enough. > Even if we can cap each app's pending request, we cannot handle the case that there're many running apps, each of them has capped but still significant numbers of pending resources. > So we may possibly need to add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org For additional commands, e-mail: yarn-issues-help@hadoop.apache.org