hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9719) Refactoring ErasureCodingWorker into smaller reusable constructs
Date Mon, 01 Feb 2016 12:57:39 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15126207#comment-15126207
] 

Hadoop QA commented on HDFS-9719:
---------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s {color} | {color:blue}
Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green}
The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color}
| {color:green} The patch appears to include 1 new or modified test files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 21s {color} | {color:blue}
Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 28s {color}
| {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 13s {color} |
{color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 19s {color} |
{color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 30s {color}
| {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 27s {color} |
{color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 26s {color}
| {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 43s {color} |
{color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 27s {color} |
{color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 15s {color} |
{color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 17s {color} | {color:blue}
Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 20s {color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 27s {color} |
{color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 3m 5s {color} | {color:red}
hadoop-hdfs-project-jdk1.8.0_66 with JDK v1.8.0_66 generated 1 new + 49 unchanged - 0 fixed
= 50 total (was 49) {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 27s {color} | {color:green}
the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 21s {color} |
{color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 4m 26s {color} | {color:red}
hadoop-hdfs-project-jdk1.7.0_91 with JDK v1.7.0_91 generated 1 new + 51 unchanged - 0 fixed
= 52 total (was 51) {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 21s {color} | {color:green}
the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 28s {color} | {color:red}
hadoop-hdfs-project: patch generated 95 new + 291 unchanged - 32 fixed = 386 total (was 323)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 21s {color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 22s {color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color}
| {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 11s {color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 18s {color} |
{color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 6m 4s {color} | {color:red}
hadoop-hdfs-project_hadoop-hdfs-client-jdk1.7.0_91 with JDK v1.7.0_91 generated 1 new + 1
unchanged - 0 fixed = 2 total (was 1) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 7s {color} | {color:green}
the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 49s {color} | {color:green}
hadoop-hdfs-client in the patch passed with JDK v1.8.0_66. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 62m 56s {color} | {color:red}
hadoop-hdfs in the patch failed with JDK v1.8.0_66. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 56s {color} | {color:green}
hadoop-hdfs-client in the patch passed with JDK v1.7.0_91. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 57m 25s {color} | {color:green}
hadoop-hdfs in the patch passed with JDK v1.7.0_91. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 21s {color}
| {color:green} Patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 158m 22s {color} | {color:black}
{color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_66 Failed junit tests | hadoop.hdfs.TestFileAppend |
|   | hadoop.hdfs.server.datanode.TestBlockScanner |
|   | hadoop.hdfs.server.namenode.TestNamenodeCapacityReport |
| JDK v1.8.0_66 Timed out junit tests | org.apache.hadoop.hdfs.TestLeaseRecovery2 |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |  Image:yetus/hadoop:0ca8df7 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12785484/HDFS-9719-v1.patch
|
| JIRA Issue | HDFS-9719 |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  unit  findbugs
 checkstyle  |
| uname | Linux 6e1f4555bc94 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12
UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 8f2622b |
| Default Java | 1.7.0_91 |
| Multi-JDK versions |  /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
|
| findbugs | v3.0.0 |
| javac | hadoop-hdfs-project-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-HDFS-Build/14315/artifact/patchprocess/diff-compile-javac-hadoop-hdfs-project-jdk1.8.0_66.txt
|
| javac | hadoop-hdfs-project-jdk1.7.0_91: https://builds.apache.org/job/PreCommit-HDFS-Build/14315/artifact/patchprocess/diff-compile-javac-hadoop-hdfs-project-jdk1.7.0_91.txt
|
| checkstyle | https://builds.apache.org/job/PreCommit-HDFS-Build/14315/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt
|
| javadoc | hadoop-hdfs-project_hadoop-hdfs-client-jdk1.7.0_91: https://builds.apache.org/job/PreCommit-HDFS-Build/14315/artifact/patchprocess/diff-javadoc-javadoc-hadoop-hdfs-project_hadoop-hdfs-client-jdk1.7.0_91.txt
|
| unit | https://builds.apache.org/job/PreCommit-HDFS-Build/14315/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
|
| unit test logs |  https://builds.apache.org/job/PreCommit-HDFS-Build/14315/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
|
| JDK v1.7.0_91  Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/14315/testReport/
|
| modules | C:  hadoop-hdfs-project/hadoop-hdfs-client   hadoop-hdfs-project/hadoop-hdfs 
U: hadoop-hdfs-project |
| Max memory used | 77MB |
| Powered by | Apache Yetus 0.2.0-SNAPSHOT   http://yetus.apache.org |
| Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/14315/console |


This message was automatically generated.



> Refactoring ErasureCodingWorker into smaller reusable constructs
> ----------------------------------------------------------------
>
>                 Key: HDFS-9719
>                 URL: https://issues.apache.org/jira/browse/HDFS-9719
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Kai Zheng
>            Assignee: Kai Zheng
>         Attachments: HDFS-9719-v1.patch
>
>
> This would suggest and refactor {{ErasureCodingWorker}} into smaller constructs to be
reused in other places like block group checksum computing in datanode side. As discussed
in HDFS-8430 and implemented in HDFS-9694 patch, checksum computing for striped block groups
would be distributed to datanode in the group, where data block data should be able to be
reconstructed when missed/corrupted to recompute the block checksum. The most needed codes
are in the current ErasureCodingWorker and could be reused in order to avoid duplication.
Fortunately, we have very good and complete tests, which would make the refactoring much easier.
The refactoring will also help a lot for subsequent tasks in phase II for non-striping erasure
coded files and blocks. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message