hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kihwal Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9318) considerLoad factor can be improved
Date Mon, 02 Nov 2015 23:26:27 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14986263#comment-14986263
] 

Kihwal Lee commented on HDFS-9318:
----------------------------------

The precommit ran but failed to post the result because of this error: {{JIRA Plugin: no credentials
provided to write a comment.}}
https://builds.apache.org/job/PreCommit-HDFS-Build/13243/console

Here is the result:




-1 overall

|| Vote ||      Subsystem ||  Runtime   || Comment ||
|   0  |        reexec  |  0m 10s    | docker + precommit patch detected. 
|  +1  |       @author  |  0m 0s     | The patch does not contain any @author 
|      |                |            | tags.
|  +1  |    test4tests  |  0m 0s     | The patch appears to include 1 new or 
|      |                |            | modified test files.
|  +1  |    mvninstall  |  3m 17s    | trunk passed 
|  +1  |       compile  |  0m 35s    | trunk passed with JDK v1.8.0_60 
|  +1  |       compile  |  0m 33s    | trunk passed with JDK v1.7.0_79 
|  +1  |    checkstyle  |  0m 15s    | trunk passed 
|  +1  |    mvneclipse  |  0m 15s    | trunk passed 
|  -1  |      findbugs  |  1m 58s    | hadoop-hdfs-project/hadoop-hdfs in trunk 
|      |                |            | cannot run convertXmlToText from findbugs
|  +1  |       javadoc  |  1m 11s    | trunk passed with JDK v1.8.0_60 
|  +1  |       javadoc  |  1m 52s    | trunk passed with JDK v1.7.0_79 
|  +1  |    mvninstall  |  0m 40s    | the patch passed 
|  +1  |       compile  |  0m 33s    | the patch passed with JDK v1.8.0_60 
|  +1  |         javac  |  0m 33s    | the patch passed 
|  +1  |       compile  |  0m 33s    | the patch passed with JDK v1.7.0_79 
|  +1  |         javac  |  0m 33s    | the patch passed 
|  -1  |    checkstyle  |  0m 16s    | Patch generated 1 new checkstyle issues 
|      |                |            | in hadoop-hdfs-project/hadoop-hdfs (total
|      |                |            | was 426, now 427).
|  +1  |    mvneclipse  |  0m 15s    | the patch passed 
|  +1  |    whitespace  |  0m 0s     | Patch has no whitespace issues. 
|  +1  |      findbugs  |  2m 9s     | the patch passed 
|  +1  |       javadoc  |  1m 11s    | the patch passed with JDK v1.8.0_60 
|  +1  |       javadoc  |  1m 53s    | the patch passed with JDK v1.7.0_79 
|  -1  |          unit  |  56m 43s   | hadoop-hdfs in the patch failed with JDK 
|      |                |            | v1.8.0_60.
|  -1  |          unit  |  52m 17s   | hadoop-hdfs in the patch failed with JDK 
|      |                |            | v1.7.0_79.
|  -1  |    asflicense  |  0m 19s    | Patch generated 56 ASF License warnings. 
|      |                |  129m 48s  | 


|| Reason || Tests ||
| JDK v1.7.0_79 Failed junit tests  |  hadoop.hdfs.server.blockmanagement.TestNodeCount 
|                                   |  hadoop.hdfs.server.datanode.TestBlockScanner 
|                                   |  hadoop.hdfs.server.datanode.fsdataset.impl.TestScrLazyPersistFiles

|                                   |  hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock

|                                   |  hadoop.hdfs.util.TestByteArrayManager 
|                                   |  hadoop.hdfs.TestDFSStripedOutputStreamWithFailure 
|                                   |  hadoop.hdfs.TestDFSStripedOutputStreamWithFailure010



|| Subsystem || Report/Notes ||
| Docker | Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-10-28 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12769181/HDFS-9318-v1.patch
|
| JIRA Issue | HDFS-9318 |
| Optional Tests |  asflicense  javac  javadoc  mvninstall  unit  findbugs  checkstyle  compile
 |
| uname | Linux 670c555545bc 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12
UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/apache-yetus-06655ab/dev-support/personality/hadoop.sh
|
| git revision | trunk / 6ff6663 |
| Default Java | 1.7.0_79 |
| Multi-JDK versions |  /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
|
| findbugs | v3.0.0 |
| findbugs | /testptch/hadoop/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs.html
|
| checkstyle | /testptch/hadoop/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
|
| unit | /testptch/hadoop/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_60.txt
|
| unit | /testptch/hadoop/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt
|
| unit test logs |  /testptch/hadoop/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_60.txt
/testptch/hadoop/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt |
| JDK v1.7.0_79  Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/13243/testReport/
|
| asflicense | /testptch/hadoop/patchprocess/patch-asflicense-problems.txt |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
| Max memory used | 223MB |
| Powered by | Apache Yetus   http://yetus.apache.org |
| Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/13243/console |


> considerLoad factor can be improved
> -----------------------------------
>
>                 Key: HDFS-9318
>                 URL: https://issues.apache.org/jira/browse/HDFS-9318
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.6.0
>            Reporter: Kuhu Shukla
>            Assignee: Kuhu Shukla
>         Attachments: HDFS-9318-v1.patch
>
>
> Currently considerLoad avoids choosing nodes that are too active, so it helps level the
HDFS load across the cluster. Under normal conditions, this is desired. However, when a cluster
has a large percentage of nearly full nodes, this can make it difficult to find good targets
because the placement policy wants to avoid the full nodes, but considerLoad wants to avoid
the busy less-full nodes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message