hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14553) Add (parallelized) integration tests to hadoop-azure
Date Wed, 16 Aug 2017 20:11:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16129340#comment-16129340
] 

Hadoop QA commented on HADOOP-14553:
------------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 20s{color} | {color:blue}
Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  0s{color} |
{color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m  0s{color}
| {color:green} The patch appears to include 101 new or modified test files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 25s{color} | {color:blue}
Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m 57s{color}
| {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 18m 20s{color} |
{color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 10s{color}
| {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  2m 10s{color} |
{color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m 20s{color} |
{color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 26s{color} |
{color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 21s{color} | {color:blue}
Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 14s{color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m 31s{color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 12m 31s{color} | {color:green}
root generated 0 new + 1315 unchanged - 2 fixed = 1315 total (was 1317) {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  2m 15s{color}
| {color:orange} root: The patch generated 123 new + 212 unchanged - 178 fixed = 335 total
(was 390) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  2m 18s{color} |
{color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  0s{color} | {color:red}
The patch has 13 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>.
Refer https://git-scm.com/docs/git-apply {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  2s{color} | {color:green}
The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m 41s{color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 24s{color} |
{color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}  8m 10s{color} | {color:red}
hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  1m 17s{color} | {color:green}
hadoop-azure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 50s{color}
| {color:green} The patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 98m 52s{color} | {color:black}
{color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.ipc.TestRPC |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |  Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14553 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12882179/HADOOP-14553-014.patch
|
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  unit  findbugs
 checkstyle  xml  |
| uname | Linux f00f3e70038e 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017
x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 1455306 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| checkstyle | https://builds.apache.org/job/PreCommit-HADOOP-Build/13049/artifact/patchprocess/diff-checkstyle-root.txt
|
| whitespace | https://builds.apache.org/job/PreCommit-HADOOP-Build/13049/artifact/patchprocess/whitespace-eol.txt
|
| unit | https://builds.apache.org/job/PreCommit-HADOOP-Build/13049/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
|
|  Test Results | https://builds.apache.org/job/PreCommit-HADOOP-Build/13049/testReport/ |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-azure U: . |
| Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/13049/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT   http://yetus.apache.org |


This message was automatically generated.



> Add (parallelized) integration tests to hadoop-azure
> ----------------------------------------------------
>
>                 Key: HADOOP-14553
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14553
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/azure
>    Affects Versions: 2.9.0
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>         Attachments: HADOOP-14553-001.patch, HADOOP-14553-002.patch, HADOOP-14553-003.patch,
HADOOP-14553-004.patch, HADOOP-14553-005.patch, HADOOP-14553-006.patch, HADOOP-14553-007.patch,
HADOOP-14553-008.patch, HADOOP-14553-009.patch, HADOOP-14553-010.patch, HADOOP-14553-011.patch,
HADOOP-14553-012.patch, HADOOP-14553-014.patch
>
>
> The Azure tests are slow to run as they are serialized, as they are all called Test*
there's no clear differentiation from unit tests which Jenkins can run, and integration tests
which it can't.
> Move the azure tests {{Test*}} to integration tests {{ITest*}}, parallelize (which includes
having separate paths for every test suite). The code in hadoop-aws's POM &c show what
to do.
> *UPDATE August 4, 2017*:  Adding a list of requirements to clarify the acceptance criteria
for this JIRA:
> # Parallelize test execution
> # Define test groups: i) UnitTests - self-contained, executed by Jenkins, ii) IntegrationTests
- requires Azure Storage account, executed by engineers prior to check-in, and if needed,
iii)	ScaleTests – long running performance and scalability tests.
> # Define configuration profiles to run tests with different settings.  Allows an engineer
to run “IntegrationTests” with fs.azure.secure.mode = true and false.  Need to review
settings to see what else would benefit.
> # Maven commands to run b) and c).  Turns out it is not easy to do with Maven, so we
might have to run it multiple times to run with different configuration settings.
> # Document how to add and run tests and the process for contributing to Apache Hadoop.
 Steve shared an example at https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/testing.md

> # UnitTests should run in under 2 minutes and IntegrationTests should run in under 15
minutes, even on slower network connections.  (These are rough goals)
> # Ensure test data (containers/blobs/etc) is deleted.  Exceptions for large persistent
content used repeatedly to expedite test execution. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message