hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-12345) Scale testing HDFS NameNode with real metadata and workloads (Dynamometer)
Date Thu, 14 Feb 2019 03:16:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16767849#comment-16767849
] 

Hadoop QA commented on HDFS-12345:
----------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 15s{color} | {color:blue}
Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  0s{color} |
{color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m  0s{color}
| {color:green} The patch appears to include 19 new or modified test files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  1m 11s{color} | {color:blue}
Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m 20s{color}
| {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m 28s{color} |
{color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 55s{color}
| {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 13m 42s{color} |
{color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 11m 25s{color}
| {color:green} branch has no errors when building and testing our client artifacts. {color}
|
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  0s{color} | {color:blue}
Skipped patched modules with no Java source: hadoop-assemblies hadoop-tools hadoop-dist .
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m  0s{color} |
{color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  5m 20s{color} |
{color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 31s{color} | {color:blue}
Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m  3s{color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m 38s{color} |
{color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 14m 38s{color} | {color:red}
root generated 15 new + 1495 unchanged - 0 fixed = 1510 total (was 1495) {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  3m 23s{color}
| {color:orange} root: The patch generated 865 new + 0 unchanged - 0 fixed = 865 total (was
0) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 12m 49s{color} |
{color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} shellcheck {color} | {color:red}  0m  2s{color} | {color:red}
The patch generated 87 new + 1 unchanged - 0 fixed = 88 total (was 1) {color} |
| {color:orange}-0{color} | {color:orange} shelldocs {color} | {color:orange}  0m 13s{color}
| {color:orange} The patch generated 4 new + 104 unchanged - 0 fixed = 108 total (was 104)
{color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  0s{color} | {color:red}
The patch has 8 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>.
Refer https://git-scm.com/docs/git-apply {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  1s{color} | {color:red}
The patch 6 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m 22s{color} | {color:green}
The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 11m 22s{color}
| {color:green} patch has no errors when building and testing our client artifacts. {color}
|
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  0s{color} | {color:blue}
Skipped patched modules with no Java source: hadoop-assemblies hadoop-tools/hadoop-dynamometer
hadoop-tools hadoop-dist . hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-dist {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m 23s{color} |
{color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red}  5m 39s{color} | {color:red}
root generated 1 new + 4202 unchanged - 0 fixed = 4203 total (was 4202) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 14m 40s{color} | {color:red}
root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red}  0m 48s{color} | {color:red}
The patch generated 7 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}158m 27s{color} | {color:black}
{color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.util.TestReadWriteDiskValidator |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HDFS-12345 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12958638/HDFS-12345.001.patch
|
| Optional Tests |  dupname  asflicense  shellcheck  shelldocs  compile  javac  javadoc  mvninstall
 mvnsite  unit  shadedclient  xml  findbugs  checkstyle  |
| uname | Linux dfc12830ab5f 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018
x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / fa067aa |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| shellcheck | v0.4.6 |
| javac | https://builds.apache.org/job/PreCommit-HDFS-Build/26213/artifact/out/diff-compile-javac-root.txt
|
| checkstyle | https://builds.apache.org/job/PreCommit-HDFS-Build/26213/artifact/out/diff-checkstyle-root.txt
|
| shellcheck | https://builds.apache.org/job/PreCommit-HDFS-Build/26213/artifact/out/diff-patch-shellcheck.txt
|
| shelldocs | https://builds.apache.org/job/PreCommit-HDFS-Build/26213/artifact/out/diff-patch-shelldocs.txt
|
| whitespace | https://builds.apache.org/job/PreCommit-HDFS-Build/26213/artifact/out/whitespace-eol.txt
|
| whitespace | https://builds.apache.org/job/PreCommit-HDFS-Build/26213/artifact/out/whitespace-tabs.txt
|
| javadoc | https://builds.apache.org/job/PreCommit-HDFS-Build/26213/artifact/out/diff-javadoc-javadoc-root.txt
|
| unit | https://builds.apache.org/job/PreCommit-HDFS-Build/26213/artifact/out/patch-unit-root.txt
|
|  Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/26213/testReport/ |
| asflicense | https://builds.apache.org/job/PreCommit-HDFS-Build/26213/artifact/out/patch-asflicense-problems.txt
|
| Max. process+thread count | 1415 (vs. ulimit of 10000) |
| modules | C: hadoop-assemblies hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-workload
hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-blockgen hadoop-tools/hadoop-dynamometer
hadoop-tools hadoop-dist . hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-dist hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-infra
U: . |
| Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/26213/console |
| Powered by | Apache Yetus 0.8.0   http://yetus.apache.org |


This message was automatically generated.



> Scale testing HDFS NameNode with real metadata and workloads (Dynamometer)
> --------------------------------------------------------------------------
>
>                 Key: HDFS-12345
>                 URL: https://issues.apache.org/jira/browse/HDFS-12345
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: namenode, test
>            Reporter: Zhe Zhang
>            Assignee: Erik Krogen
>            Priority: Major
>         Attachments: HDFS-12345.000.patch, HDFS-12345.001.patch
>
>
> Dynamometer has now been open sourced on our [GitHub page|https://github.com/linkedin/dynamometer].
Read more at our [recent blog post|https://engineering.linkedin.com/blog/2018/02/dynamometer--scale-testing-hdfs-on-minimal-hardware-with-maximum].
> To encourage getting the tool into the open for others to use as quickly as possible,
we went through our standard open sourcing process of releasing on GitHub. However we are
interested in the possibility of donating this to Apache as part of Hadoop itself and would
appreciate feedback on whether or not this is something that would be supported by the community.
> Also of note, previous [discussions on the dev mail lists|http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201707.mbox/%3c98FCEFFA-FAFF-4CF1-A14D-4FAAB6567F31@gmail.com%3e]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message