hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4844) Add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource
Date Tue, 17 May 2016 22:37:13 GMT

    [ https://issues.apache.org/jira/browse/YARN-4844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15287785#comment-15287785
] 

Hadoop QA commented on YARN-4844:
---------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 13s {color} | {color:blue}
Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green}
The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color}
| {color:green} The patch appears to include 55 new or modified test files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 19s {color} | {color:blue}
Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m 19s {color}
| {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m 15s {color} |
{color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m 4s {color}
| {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 5m 15s {color} |
{color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m 18s {color}
| {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 9m 33s {color} |
{color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m 28s {color} |
{color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 21s {color} | {color:blue}
Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 1m 2s {color} | {color:red}
hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 24s {color} | {color:red}
hadoop-sls in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 8m 45s {color} | {color:red}
root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} cc {color} | {color:red} 8m 45s {color} | {color:red}
root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 8m 45s {color} | {color:red}
root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 3m 33s {color} | {color:red}
root: patch generated 192 new + 4323 unchanged - 117 fixed = 4515 total (was 4440) {color}
|
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 1m 4s {color} | {color:red}
hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m 23s {color} | {color:red}
hadoop-sls in the patch failed. {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m 18s {color}
| {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s {color} | {color:red}
The patch has 1 line(s) with tabs. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 48s {color} | {color:red}
hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 22s {color} | {color:red}
hadoop-sls in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m 31s {color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 40s {color} | {color:green}
hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m 21s {color} | {color:green}
hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 40s {color} | {color:green}
hadoop-yarn-server-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 13m 51s {color} | {color:red}
hadoop-yarn-server-nodemanager in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 58s {color} | {color:red}
hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 1m 48s {color} | {color:red}
hadoop-yarn-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 46s {color} | {color:red}
hadoop-yarn-applications-distributedshell in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 23s {color} | {color:red}
hadoop-sls in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 34s {color}
| {color:green} Patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 118m 46s {color} | {color:black}
{color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainerMetrics
|
|   | hadoop.yarn.client.TestApplicationMasterServiceProtocolOnHA |
|   | hadoop.yarn.client.TestGetGroups |
|   | hadoop.yarn.client.TestHedgingRequestRMFailoverProxyProvider |
|   | hadoop.yarn.client.TestRMFailover |
|   | hadoop.yarn.client.TestResourceManagerAdministrationProtocolPBClientImpl |
|   | hadoop.yarn.client.TestApplicationClientProtocolOnHA |
|   | hadoop.yarn.client.api.impl.TestAMRMClient |
|   | hadoop.yarn.client.api.impl.TestNMClient |
|   | hadoop.yarn.client.api.impl.TestAMRMClientOnRMRestart |
|   | hadoop.yarn.client.cli.TestYarnCLI |
|   | hadoop.yarn.client.api.impl.TestAMRMProxy |
|   | hadoop.yarn.client.TestResourceTrackerOnHA |
|   | hadoop.yarn.client.api.impl.TestYarnClient |
|   | hadoop.yarn.applications.distributedshell.TestDistributedShellWithNodeLabels |
|   | hadoop.yarn.applications.distributedshell.TestDistributedShell |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |  Image:yetus/hadoop:2c91fd8 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12804505/YARN-4844.8.patch
|
| JIRA Issue | YARN-4844 |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  unit  findbugs
 checkstyle  cc  |
| uname | Linux 1b2826501bde 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12
UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 1356cbe |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| mvninstall | https://builds.apache.org/job/PreCommit-YARN-Build/11510/artifact/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| mvninstall | https://builds.apache.org/job/PreCommit-YARN-Build/11510/artifact/patchprocess/patch-mvninstall-hadoop-tools_hadoop-sls.txt
|
| compile | https://builds.apache.org/job/PreCommit-YARN-Build/11510/artifact/patchprocess/patch-compile-root.txt
|
| cc | https://builds.apache.org/job/PreCommit-YARN-Build/11510/artifact/patchprocess/patch-compile-root.txt
|
| javac | https://builds.apache.org/job/PreCommit-YARN-Build/11510/artifact/patchprocess/patch-compile-root.txt
|
| checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/11510/artifact/patchprocess/diff-checkstyle-root.txt
|
| mvnsite | https://builds.apache.org/job/PreCommit-YARN-Build/11510/artifact/patchprocess/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| mvnsite | https://builds.apache.org/job/PreCommit-YARN-Build/11510/artifact/patchprocess/patch-mvnsite-hadoop-tools_hadoop-sls.txt
|
| whitespace | https://builds.apache.org/job/PreCommit-YARN-Build/11510/artifact/patchprocess/whitespace-tabs.txt
|
| findbugs | https://builds.apache.org/job/PreCommit-YARN-Build/11510/artifact/patchprocess/patch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| findbugs | https://builds.apache.org/job/PreCommit-YARN-Build/11510/artifact/patchprocess/patch-findbugs-hadoop-tools_hadoop-sls.txt
|
| unit | https://builds.apache.org/job/PreCommit-YARN-Build/11510/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
|
| unit | https://builds.apache.org/job/PreCommit-YARN-Build/11510/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| unit | https://builds.apache.org/job/PreCommit-YARN-Build/11510/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
|
| unit | https://builds.apache.org/job/PreCommit-YARN-Build/11510/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-applications-distributedshell.txt
|
| unit | https://builds.apache.org/job/PreCommit-YARN-Build/11510/artifact/patchprocess/patch-unit-hadoop-tools_hadoop-sls.txt
|
| unit test logs |  https://builds.apache.org/job/PreCommit-YARN-Build/11510/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
https://builds.apache.org/job/PreCommit-YARN-Build/11510/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
https://builds.apache.org/job/PreCommit-YARN-Build/11510/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-applications-distributedshell.txt
|
|  Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/11510/testReport/ |
| modules | C:  hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api   hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
  hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common   hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
  hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager  
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client   hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell
  hadoop-tools/hadoop-sls  U: . |
| Console output | https://builds.apache.org/job/PreCommit-YARN-Build/11510/console |
| Powered by | Apache Yetus 0.2.0   http://yetus.apache.org |


This message was automatically generated.



> Add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource
> --------------------------------------------------------------------
>
>                 Key: YARN-4844
>                 URL: https://issues.apache.org/jira/browse/YARN-4844
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: api
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>            Priority: Blocker
>         Attachments: YARN-4844.1.patch, YARN-4844.2.patch, YARN-4844.3.patch, YARN-4844.4.patch,
YARN-4844.5.patch, YARN-4844.6.patch, YARN-4844.7.patch, YARN-4844.8.branch-2.patch, YARN-4844.8.patch
>
>
> We use int32 for memory now, if a cluster has 10k nodes, each node has 210G memory, we
will get a negative total cluster memory.
> And another case that easier overflows int32 is: we added all pending resources of running
apps to cluster's total pending resources. If a problematic app requires too much resources
(let's say 1M+ containers, each of them has 3G containers), int32 will be not enough.
> Even if we can cap each app's pending request, we cannot handle the case that there're
many running apps, each of them has capped but still significant numbers of pending resources.
> So we may possibly need to add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message