maven-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] galderz opened a new pull request #221: Support running tests until failure or N test runs
Date Wed, 13 Feb 2019 17:03:02 GMT
galderz opened a new pull request #221: Support running tests until failure or N test runs
URL: https://github.com/apache/maven-surefire/pull/221
 
 
   This is an initial proposal to add surefire/failsafe property called `untilFailureLoopCount`.
   
   I often find myself in the situation where there's a random test failure happening in CI,
and have to run the one or several tests continuously until it fails, maybe running them with
TRACE logging.
   
   Normally, I'd wrap the `mvn test...` call with a script that calls it in a loop, but this
can be quite "slow" since each iteration is a new JVM launch...etc.
   
   The idea behind `untilFailureLoopCount` is that you've give it a positive number, say 1000,
and the test(s) would run continuously until either a failure happens or the test is executed
that many times.
   
   I've created [this randomly failing test](https://github.com/galderz/surefire-until-failure-loop-count/blob/master/src/test/java/com/acme/maven/surefire/untilfailure/UntilFailureLoopCountTest.java)
and I've run it with `mvn -Dsurefire.untilFailureLoopCount=10 test` and the output would be
something like:
   
   ```
   [INFO] -------------------------------------------------------
   [INFO]  T E S T S
   [INFO] -------------------------------------------------------
   [INFO] Running com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
   [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.034 s - in com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
   [INFO] Running com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
   [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 s - in com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
   [INFO] Running com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
   [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
   [INFO] Running com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
   [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
   [INFO] Running com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
   [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.001 s <<<
FAILURE! - in com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
   [ERROR] com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest.testEventualFailure
 Time elapsed: 0.001 s  <<< FAILURE!
   java.lang.AssertionError: Just messing with your testing
   	at com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest.testEventualFailure(UntilFailureLoopCountTest.java:16)
   ```
   
   If maintainers are interested in this, I can complete the PR with tests, documentation
and support for other providers. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message