hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4844) Add getMemorySize/getVirtualCoresSize to o.a.h.y.api.records.Resource
Date Wed, 01 Jun 2016 16:10:59 GMT

    [ https://issues.apache.org/jira/browse/YARN-4844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15310552#comment-15310552
] 

Hadoop QA commented on YARN-4844:
---------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 26s {color} | {color:blue}
Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green}
The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color}
| {color:green} The patch appears to include 59 new or modified test files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m 27s {color} | {color:blue}
Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 25s {color}
| {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 21s {color} |
{color:green} branch-2.8 passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 56s {color} |
{color:green} branch-2.8 passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 38s {color}
| {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 5m 13s {color} |
{color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m 48s {color}
| {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 9m 55s {color} |
{color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 38s {color} |
{color:green} branch-2.8 passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 52s {color} |
{color:green} branch-2.8 passed with JDK v1.7.0_101 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 14s {color} | {color:blue}
Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m 23s {color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 9s {color} | {color:green}
the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 6m 9s {color} | {color:green}
the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 9s {color} | {color:green}
the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 43s {color} |
{color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 6m 43s {color} | {color:green}
the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 6m 43s {color} | {color:red}
root-jdk1.7.0_101 with JDK v1.7.0_101 generated 3 new + 895 unchanged - 0 fixed = 898 total
(was 895) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m 35s {color} | {color:red}
root: The patch generated 91 new + 2322 unchanged - 77 fixed = 2413 total (was 2399) {color}
|
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 5m 10s {color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m 31s {color}
| {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s {color} | {color:red}
The patch has 6 line(s) that end in whitespace. Use git apply --whitespace=fix. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s {color} | {color:red}
The patch 1 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 11m 46s {color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 37s {color} |
{color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 15s {color} |
{color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 25s {color} | {color:green}
hadoop-yarn-api in the patch passed with JDK v1.8.0_91. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 9s {color} | {color:green}
hadoop-yarn-common in the patch passed with JDK v1.8.0_91. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 22s {color} | {color:green}
hadoop-yarn-server-common in the patch passed with JDK v1.8.0_91. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 9m 2s {color} | {color:red}
hadoop-yarn-server-nodemanager in the patch failed with JDK v1.8.0_91. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 73m 0s {color} | {color:red}
hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_91. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 65m 39s {color} | {color:red}
hadoop-yarn-client in the patch failed with JDK v1.8.0_91. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m 16s {color} | {color:green}
hadoop-yarn-applications-distributedshell in the patch passed with JDK v1.8.0_91. {color}
|
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 48s {color} | {color:green}
hadoop-mapreduce-client-core in the patch passed with JDK v1.8.0_91. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 40s {color} | {color:green}
hadoop-mapreduce-client-common in the patch passed with JDK v1.8.0_91. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m 12s {color} | {color:green}
hadoop-mapreduce-client-app in the patch passed with JDK v1.8.0_91. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 99m 39s {color} | {color:green}
hadoop-mapreduce-client-jobclient in the patch passed with JDK v1.8.0_91. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 58s {color} | {color:green}
hadoop-sls in the patch passed with JDK v1.8.0_91. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 32s {color} | {color:green}
hadoop-yarn-api in the patch passed with JDK v1.7.0_101. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 29s {color} | {color:green}
hadoop-yarn-common in the patch passed with JDK v1.7.0_101. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 34s {color} | {color:green}
hadoop-yarn-server-common in the patch passed with JDK v1.7.0_101. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 9m 41s {color} | {color:red}
hadoop-yarn-server-nodemanager in the patch failed with JDK v1.7.0_101. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 74m 37s {color} | {color:red}
hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_101. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 66m 3s {color} | {color:red}
hadoop-yarn-client in the patch failed with JDK v1.7.0_101. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m 39s {color} | {color:green}
hadoop-yarn-applications-distributedshell in the patch passed with JDK v1.7.0_101. {color}
|
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 21s {color} | {color:green}
hadoop-mapreduce-client-core in the patch passed with JDK v1.7.0_101. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 57s {color} | {color:green}
hadoop-mapreduce-client-common in the patch passed with JDK v1.7.0_101. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 10m 17s {color} | {color:red}
hadoop-mapreduce-client-app in the patch failed with JDK v1.7.0_101. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 94m 26s {color} | {color:red}
hadoop-mapreduce-client-jobclient in the patch failed with JDK v1.7.0_101. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 29s {color} | {color:red}
hadoop-sls in the patch failed with JDK v1.7.0_101. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 22s {color}
| {color:green} The patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 641m 24s {color} | {color:black}
{color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_91 Failed junit tests | hadoop.yarn.server.nodemanager.containermanager.TestNMProxy
|
|   | hadoop.yarn.server.nodemanager.containermanager.localizer.TestResourceLocalizationService
|
|   | hadoop.yarn.server.resourcemanager.TestClientRMTokens |
|   | hadoop.yarn.server.resourcemanager.TestAMAuthorization |
|   | hadoop.yarn.client.api.impl.TestAMRMProxy |
|   | hadoop.yarn.client.TestGetGroups |
| JDK v1.8.0_91 Timed out junit tests | org.apache.hadoop.yarn.client.cli.TestYarnCLI |
|   | org.apache.hadoop.yarn.client.api.impl.TestYarnClient |
|   | org.apache.hadoop.yarn.client.api.impl.TestAMRMClient |
|   | org.apache.hadoop.yarn.client.api.impl.TestNMClient |
| JDK v1.7.0_101 Failed junit tests | hadoop.mapreduce.lib.jobcontrol.TestMapReduceJobControl
|
\\
\\
|| Subsystem || Report/Notes ||
| Docker |  Image:yetus/hadoop:c60792e |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12807330/YARN-4844-branch-2.8.0016.patch
|
| JIRA Issue | YARN-4844 |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  unit  findbugs
 checkstyle  cc  |
| uname | Linux f1f10b49018a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12
UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
| git revision | branch-2.8 / 2f3e1d9 |
| Default Java | 1.7.0_101 |
| Multi-JDK versions |  /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101
|
| findbugs | v3.0.0 |
| javac | https://builds.apache.org/job/PreCommit-YARN-Build/11784/artifact/patchprocess/diff-compile-javac-root-jdk1.7.0_101.txt
|
| checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/11784/artifact/patchprocess/diff-checkstyle-root.txt
|
| whitespace | https://builds.apache.org/job/PreCommit-YARN-Build/11784/artifact/patchprocess/whitespace-eol.txt
|
| whitespace | https://builds.apache.org/job/PreCommit-YARN-Build/11784/artifact/patchprocess/whitespace-tabs.txt
|
| unit | https://builds.apache.org/job/PreCommit-YARN-Build/11784/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdk1.8.0_91.txt
|
| unit | https://builds.apache.org/job/PreCommit-YARN-Build/11784/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt
|
| unit | https://builds.apache.org/job/PreCommit-YARN-Build/11784/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.8.0_91.txt
|
| unit | https://builds.apache.org/job/PreCommit-YARN-Build/11784/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdk1.7.0_101.txt
|
| unit | https://builds.apache.org/job/PreCommit-YARN-Build/11784/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt
|
| unit | https://builds.apache.org/job/PreCommit-YARN-Build/11784/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_101.txt
|
| unit | https://builds.apache.org/job/PreCommit-YARN-Build/11784/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_101.txt
|
| unit | https://builds.apache.org/job/PreCommit-YARN-Build/11784/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.7.0_101.txt
|
| unit | https://builds.apache.org/job/PreCommit-YARN-Build/11784/artifact/patchprocess/patch-unit-hadoop-tools_hadoop-sls-jdk1.7.0_101.txt
|
| unit test logs |  https://builds.apache.org/job/PreCommit-YARN-Build/11784/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdk1.8.0_91.txt
https://builds.apache.org/job/PreCommit-YARN-Build/11784/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt
https://builds.apache.org/job/PreCommit-YARN-Build/11784/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.8.0_91.txt
https://builds.apache.org/job/PreCommit-YARN-Build/11784/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.7.0_101.txt
|
| JDK v1.7.0_101  Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/11784/testReport/
|
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
hadoop-tools/hadoop-sls U: . |
| Console output | https://builds.apache.org/job/PreCommit-YARN-Build/11784/console |
| Powered by | Apache Yetus 0.3.0   http://yetus.apache.org |


This message was automatically generated.



> Add getMemorySize/getVirtualCoresSize to o.a.h.y.api.records.Resource
> ---------------------------------------------------------------------
>
>                 Key: YARN-4844
>                 URL: https://issues.apache.org/jira/browse/YARN-4844
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: api
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>            Priority: Blocker
>             Fix For: 2.8.0
>
>         Attachments: YARN-4844-branch-2.8.0016.patch, YARN-4844-branch-2.addendum.1.patch,
YARN-4844.1.patch, YARN-4844.10.patch, YARN-4844.11.patch, YARN-4844.12.patch, YARN-4844.13.patch,
YARN-4844.14.patch, YARN-4844.15.patch, YARN-4844.16.branch-2.patch, YARN-4844.16.patch, YARN-4844.2.patch,
YARN-4844.3.patch, YARN-4844.4.patch, YARN-4844.5.patch, YARN-4844.6.patch, YARN-4844.7.patch,
YARN-4844.8.branch-2.patch, YARN-4844.8.patch, YARN-4844.9.branch, YARN-4844.9.branch-2.patch
>
>
> We use int32 for memory now, if a cluster has 10k nodes, each node has 210G memory, we
will get a negative total cluster memory.
> And another case that easier overflows int32 is: we added all pending resources of running
apps to cluster's total pending resources. If a problematic app requires too much resources
(let's say 1M+ containers, each of them has 3G containers), int32 will be not enough.
> Even if we can cap each app's pending request, we cannot handle the case that there're
many running apps, each of them has capped but still significant numbers of pending resources.
> So we may possibly need to add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message