hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7079) to support nodemanager ports management
Date Thu, 24 Aug 2017 14:59:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16140127#comment-16140127
] 

Hadoop QA commented on YARN-7079:
---------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 19s{color} | {color:blue}
Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  0s{color} |
{color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m  0s{color}
| {color:green} The patch appears to include 7 new or modified test files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 16s{color} | {color:blue}
Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m 48s{color}
| {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m 18s{color} |
{color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m  6s{color}
| {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  3m 30s{color} |
{color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  0m 51s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager in trunk
has 1 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  2m 55s{color} |
{color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 17s{color} | {color:blue}
Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  2m 46s{color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m 44s{color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 10m 44s{color} | {color:green}
the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 10m 44s{color} | {color:red}
root generated 13 new + 1291 unchanged - 0 fixed = 1304 total (was 1291) {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  2m 12s{color}
| {color:orange} root: The patch generated 171 new + 572 unchanged - 0 fixed = 743 total (was
572) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  3m 50s{color} |
{color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  0s{color} | {color:red}
The patch has 66 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>.
Refer https://git-scm.com/docs/git-apply {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  1m 32s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common generated 9 new + 0 unchanged - 0 fixed
= 9 total (was 0) {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  1m 18s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common generated 3 new
+ 0 unchanged - 0 fixed = 3 total (was 0) {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  1m  7s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager generated
1 new + 1 unchanged - 0 fixed = 2 total (was 1) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red}  0m 30s{color} | {color:red}
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api generated 4 new + 123 unchanged - 0 fixed
= 127 total (was 123) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red}  0m 33s{color} | {color:red}
hadoop-yarn-server-resourcemanager in the patch failed. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}  0m 38s{color} | {color:red}
hadoop-yarn-api in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}  2m 34s{color} | {color:red}
hadoop-yarn-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}  1m 48s{color} | {color:red}
hadoop-yarn-server-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 13m 36s{color} | {color:green}
hadoop-yarn-server-nodemanager in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 43m 38s{color} | {color:red}
hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}  7m 53s{color} | {color:red}
hadoop-sls in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red}  0m 35s{color} | {color:red}
The patch generated 4 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}167m 42s{color} | {color:black}
{color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common |
|  |  Inconsistent synchronization of org.apache.hadoop.yarn.api.records.impl.pb.ValueRangesPBImpl.bitSetStore;
locked 76% of time  Unsynchronized access at ValueRangesPBImpl.java:76% of time  Unsynchronized
access at ValueRangesPBImpl.java:[line 253] |
|  |  Inconsistent synchronization of org.apache.hadoop.yarn.api.records.impl.pb.ValueRangesPBImpl.byteStoreEnable;
locked 75% of time  Unsynchronized access at ValueRangesPBImpl.java:75% of time  Unsynchronized
access at ValueRangesPBImpl.java:[line 249] |
|  |  Inconsistent synchronization of org.apache.hadoop.yarn.api.records.impl.pb.ValueRangesPBImpl.ranges;
locked 40% of time  Unsynchronized access at ValueRangesPBImpl.java:40% of time  Unsynchronized
access at ValueRangesPBImpl.java:[line 235] |
|  |  Null passed for non-null parameter of assignRanges(List) in org.apache.hadoop.yarn.api.records.impl.pb.ValueRangesPBImpl.setRangesList(List)
 Method invoked at ValueRangesPBImpl.java:of assignRanges(List) in org.apache.hadoop.yarn.api.records.impl.pb.ValueRangesPBImpl.setRangesList(List)
 Method invoked at ValueRangesPBImpl.java:[line 205] |
|  |  Unread field:ValueRangesPBImpl.java:[line 53] |
|  |  Dead store to r in org.apache.hadoop.yarn.util.PortsInfo.refreshIfNeeded(boolean)  At
PortsInfo.java:org.apache.hadoop.yarn.util.PortsInfo.refreshIfNeeded(boolean)  At PortsInfo.java:[line
65] |
|  |  Found reliance on default encoding in org.apache.hadoop.yarn.util.PortsInfo.refreshIfNeeded(boolean):in
org.apache.hadoop.yarn.util.PortsInfo.refreshIfNeeded(boolean): new java.io.InputStreamReader(InputStream)
 At PortsInfo.java:[line 51] |
|  |  The method name org.apache.hadoop.yarn.util.PortsInfo.GetAllocatedPorts(boolean) doesn't
start with a lower case letter  At PortsInfo.java:start with a lower case letter  At PortsInfo.java:[lines
76-77] |
|  |  org.apache.hadoop.yarn.util.PortsInfo.refreshIfNeeded(boolean) may fail to close stream
 At PortsInfo.java:stream  At PortsInfo.java:[line 51] |
| FindBugs | module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
|
|  |  Inconsistent synchronization of org.apache.hadoop.yarn.server.api.records.impl.pb.NodeStatusPBImpl.builder;
locked 94% of time  Unsynchronized access at NodeStatusPBImpl.java:94% of time  Unsynchronized
access at NodeStatusPBImpl.java:[line 517] |
|  |  Inconsistent synchronization of org.apache.hadoop.yarn.server.api.records.impl.pb.NodeStatusPBImpl.proto;
locked 93% of time  Unsynchronized access at NodeStatusPBImpl.java:93% of time  Unsynchronized
access at NodeStatusPBImpl.java:[line 502] |
|  |  Inconsistent synchronization of org.apache.hadoop.yarn.server.api.records.impl.pb.NodeStatusPBImpl.viaProto;
locked 93% of time  Unsynchronized access at NodeStatusPBImpl.java:93% of time  Unsynchronized
access at NodeStatusPBImpl.java:[line 502] |
| FindBugs | module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
|
|  |  Redundant nullcheck of lastUpdatePorts which is known to be null in org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl$StatusUpdaterRunnable.run()
 Redundant null check at NodeStatusUpdaterImpl.java:is known to be null in org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl$StatusUpdaterRunnable.run()
 Redundant null check at NodeStatusUpdaterImpl.java:[line 1076] |
| Failed junit tests | hadoop.yarn.conf.TestYarnConfigurationFields |
|   | hadoop.yarn.api.TestPBImplRecords |
|   | hadoop.yarn.server.api.protocolrecords.TestProtocolRecords |
|   | hadoop.yarn.server.resourcemanager.scheduler.capacity.TestLeafQueue |
|   | hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation |
|   | hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationLimits |
|   | hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationLimitsByPartition
|
|   | hadoop.yarn.server.resourcemanager.scheduler.fifo.TestFifoScheduler |
|   | hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSAppAttempt |
|   | hadoop.yarn.server.resourcemanager.reservation.TestCapacitySchedulerPlanFollower |
|   | hadoop.yarn.server.resourcemanager.scheduler.capacity.TestPortsAllocation |
|   | hadoop.yarn.sls.appmaster.TestAMSimulator |
|   | hadoop.yarn.sls.nodemanager.TestNMSimulator |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |  Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-7079 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12883317/YARN_7079.001.patch
|
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  unit  findbugs
 checkstyle  cc  |
| uname | Linux e7d4a1c437d2 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017
x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / de0cba7 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| findbugs | https://builds.apache.org/job/PreCommit-YARN-Build/17112/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html
|
| javac | https://builds.apache.org/job/PreCommit-YARN-Build/17112/artifact/patchprocess/diff-compile-javac-root.txt
|
| checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/17112/artifact/patchprocess/diff-checkstyle-root.txt
|
| whitespace | https://builds.apache.org/job/PreCommit-YARN-Build/17112/artifact/patchprocess/whitespace-eol.txt
|
| findbugs | https://builds.apache.org/job/PreCommit-YARN-Build/17112/artifact/patchprocess/new-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.html
|
| findbugs | https://builds.apache.org/job/PreCommit-YARN-Build/17112/artifact/patchprocess/new-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.html
|
| findbugs | https://builds.apache.org/job/PreCommit-YARN-Build/17112/artifact/patchprocess/new-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.html
|
| javadoc | https://builds.apache.org/job/PreCommit-YARN-Build/17112/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api.txt
|
| javadoc | https://builds.apache.org/job/PreCommit-YARN-Build/17112/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| unit | https://builds.apache.org/job/PreCommit-YARN-Build/17112/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api.txt
|
| unit | https://builds.apache.org/job/PreCommit-YARN-Build/17112/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt
|
| unit | https://builds.apache.org/job/PreCommit-YARN-Build/17112/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt
|
| unit | https://builds.apache.org/job/PreCommit-YARN-Build/17112/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| unit | https://builds.apache.org/job/PreCommit-YARN-Build/17112/artifact/patchprocess/patch-unit-hadoop-tools_hadoop-sls.txt
|
|  Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/17112/testReport/ |
| asflicense | https://builds.apache.org/job/PreCommit-YARN-Build/17112/artifact/patchprocess/patch-asflicense-problems.txt
|
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-tools/hadoop-sls
U: . |
| Console output | https://builds.apache.org/job/PreCommit-YARN-Build/17112/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT   http://yetus.apache.org |


This message was automatically generated.



>  to support nodemanager  ports management
> -----------------------------------------
>
>                 Key: YARN-7079
>                 URL: https://issues.apache.org/jira/browse/YARN-7079
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: 田娟娟
>         Attachments: YARN_7079.001.patch
>
>
> Just like the vcores and memory, ports is also  important resource information to job
allocation . So we add the ports management logic to yarn. It can meet the user jobs' ports
 request, and  never allocate two jobs(with same port requirement) to one machine.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message