hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Elek, Marton (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDDS-1735) Create separate unit and integration test executor dev-support script
Date Fri, 05 Jul 2019 00:17:00 GMT

    [ https://issues.apache.org/jira/browse/HDDS-1735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16878913#comment-16878913
] 

Elek, Marton commented on HDDS-1735:
------------------------------------

Latest patch is uploaded. ALL the shellcheck errors are fixed under hadoop-ozone/dev-support/checks.

The latest version is tested with argo based parallel test execution. Results are available
here:

[https://github.com/elek/ozone-ci/tree/master/HDDS-1735]

(The feature branch also used to investigate HDDS-1764. In some of the builds the verbosity
of the shell scripts are increased)

FTR: Findbugs errors are independent from this patch. The shell scripts are not introduced
in patch, this patch is nothing more just a cleanup/improvement on top of the existing/accepted
features.
{quote}rat.sh runs maven command in hadoop-hdds and hadoop-ozone separately, but license errors
are only checked for hadoop-ozone project
{quote}
Thanks, this is also fixed in the latest patch.

> Create separate unit and integration test executor dev-support script
> ---------------------------------------------------------------------
>
>                 Key: HDDS-1735
>                 URL: https://issues.apache.org/jira/browse/HDDS-1735
>             Project: Hadoop Distributed Data Store
>          Issue Type: Improvement
>            Reporter: Elek, Marton
>            Assignee: Elek, Marton
>            Priority: Major
>              Labels: pull-request-available
>         Attachments: Screen Shot 2019-07-02 at 3.25.33 PM.png
>
>          Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> hadoop-ozone/dev-support/checks directory contains multiple helper script to execute
different type of testing (findbugs, rat, unit, build).
> They easily define how tests should be executed, with the following contract:
>  * The problems should be printed out to the console
>  * in case of test failure a non zero exit code should be used
>  
> The tests are working well (in fact I have some experiments with executing these scripts
on k8s and argo where all the shell scripts are executed parallel) but we need some update:
>  1. Most important: the unit tests and integration tests can be separated. Integration
tests are more flaky and it's better to have a way to run only the normal unit tests
>  2. As HDDS-1115 introduced a pom.ozone.xml it's better to use them instead of the magical
"am pl hadoop-ozone-dist" trick--
>  3. To make it possible to run blockade test in containers we should use - T flag with
docker-compose
>  4. checkstyle violations are printed out to the console



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message