hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Elek, Marton (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDDS-1735) Create separate unit and integration test executor dev-support script
Date Sat, 29 Jun 2019 17:41:00 GMT

    [ https://issues.apache.org/jira/browse/HDDS-1735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16875571#comment-16875571

Elek, Marton commented on HDDS-1735:

{quote}This patch will add code that uses pom.ozone.xml, which I would like to move to hadoop-ozone-project.
It does impact my daily usage of maven, and more effort to clean up.
It's an independent question. That's the whole point to use shell scripts: In case of a project
restructure it's enough to updated the shell scripts. As of _now_ we have the pom.ozone.xml
so I think it's not a big problem to use it.

Do you use these shell scripts? Based on your comments you prefer to use plain maven commends.
If you don't use the shell script why is it a problem to improve them? The current maven build
is not affected at all....

BTW maven lifecycle model is pretty limited compared to the graph based approach of gradle.
For example if you have more than two type of tests, not just integration test and unit test,
it doesn't fit very well. Also in your command both unit tests AND integration tests are executed.
The only thing what I did is created shortcuts to make it easier to remember which maven flags
should be used. If you have better shortcuts with exactly the same behavior, feel free to
post it in a patch.

> Create separate unit and integration test executor dev-support script
> ---------------------------------------------------------------------
>                 Key: HDDS-1735
>                 URL: https://issues.apache.org/jira/browse/HDDS-1735
>             Project: Hadoop Distributed Data Store
>          Issue Type: Improvement
>            Reporter: Elek, Marton
>            Assignee: Elek, Marton
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 1h 10m
>  Remaining Estimate: 0h
> hadoop-ozone/dev-support/checks directory contains multiple helper script to execute
different type of testing (findbugs, rat, unit, build).
> They easily define how tests should be executed, with the following contract:
>  * The problems should be printed out to the console
>  * in case of test failure a non zero exit code should be used
> The tests are working well (in fact I have some experiments with executing these scripts
on k8s and argo where all the shell scripts are executed parallel) but we need some update:
>  1. Most important: the unit tests and integration tests can be separated. Integration
tests are more flaky and it's better to have a way to run only the normal unit tests
>  2. As HDDS-1115 introduced a pom.ozone.xml it's better to use them instead of the magical
"am pl hadoop-ozone-dist" trick--
>  3. To make it possible to run blockade test in containers we should use - T flag with
>  4. checkstyle violations are printed out to the console

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message