maven-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dennis Lundberg <denn...@apache.org>
Subject Re: [Vote/Proposal]: failsafe and surefire
Date Mon, 04 Jan 2010 22:13:17 GMT
I'm not familiar enough to with the code bases to judge your proposal.

What I'd like though is a new step, before the ones you listed, and that
is to release Surefire 2.5 with whatever is in trunk *first*.

Stephen Connolly wrote:
> OK,
> 
> maven-surefire-plugin is well known to everyone
> 
> it's lesser-known sister is failsafe-maven-plugin
> 
> failsafe was written (by me) to solve some of the issues of running
> integration tests with the maven lifecycle.
> 
> the lifecycle has a number of phases
> 
> * some crappy phases
> * test
> * some more crappy phases
> * pre-integration-test
> * integration-test
> * post-integration-test
> * verify
> * yet more crappy phases
> 
> surefire binds to the test phase, and by default will fail your build
> at the test phase if any of the tests it runs fail.
> 
> That is all well and good when running unit tests...
> 
> when we want to run integration tests, we usually want to set up some
> environment which we are integrating with, e.g. start a jetty server
> and deploy the current application to that server, or start a database
> server (e.g. derby) and configure the data source, etc
> 
> so we use the pre-integration-test phase to do this set-up, and we
> tidy-up in the post-integration-test phase...
> 
> when our pre-integration-test stuff is all running in the maven JVM,
> everything is hunky-dory
> (http://www.taytohunkydorys.ie/brands/hunky_dorys.asp). we bind a
> second execution of surefire:test to the integration-test phase and
> when/if the tests fail, the JVM gets killed and our integration test
> environment gets destroyed...
> 
> however, if our pre-integration-test stuff affects external processes,
> we are stuck because we have no way to tidy-up in the
> post-integration-test phase (BTW, IMHO nobody should ever run "mvn
> integration-test". 1. it's too long to type, and 2, you reallly want
> to run "mvn verify" as that will give the post-integration-test phase
> a chance to run)
> 
> failsafe solves the issue by decoupling failing the build from running
> the tests, failsafe:integration-test never fails the build, that is
> left up to failsafe:verify, and thus (as long as you never invoke a
> phase in the range [pre-integration-test,post-integration-test]) your
> tidy-up plugin configuration will always run.
> 
> So, as part of rolling the 2.5 release of surefire, I was looking into
> merging the two plugins...
> 
> My initial idea was to just move the mojo's into m-surefire-p as is,
> thus m-s-p would have three mojos: test, integration-test and verify
> 
> However, there are advantages to keeping them as separate plugins:
> 
> * Use case 1: Ignore Unit Test Failures and run the integration tests
> (I know Bob has broken the unit tests for that new feature he is
> writing, but I want to check I have not created a regression in the
> stuff I'm working on) - the verify mojo parses the xml result files to
> see if any tests failed.  If we use the same plugin, we should really
> use the same directory (e.g. surefire-reports) It does not make sense
> for surefire to have one goal report to surefire-reports and the other
> report to failsafe-reports... and it we change the directory for the
> unit tests to e.g. test-reports that could have a regression imact.
> 
> * Use case 2: Separate Reporting (as separate plugins, they generate
> reports in separate directories as before)
> 
> * Use case 3: Separate default binding... consider the case where
> pluginManagement specifies executions, with separate plugins I can
> safely define /project/build/plugins/plugin/m-surefire-p and not drag
> in the default execution of failsafe:integration-test and
> failsafe-verify
> 
> So my proposal is as follows:
> 
> 1. Move failsafe-maven-plugin to
> https://svn.apache.org/repos/asf/maven/surefire/trunk/maven-failsafe-plugin
> 
> 2. Refactor maven-surefire-plugin taking the code that is common with
> failsafe into common module
> 
> 3. Refactor maven-failsafe-plugin to use the common module.
> 
> In pseudo code SurefirePlugin would be reduced to
> 
> execute() {
>   common.runTests();
> }
> 
> Failsafe's integration-test mojo would become
> 
> execute() {
>   try {
>     common.runTests();
>   } catch (Throwable t) {
>     // ignore
>   }
> }
> 
> 4. Refactor maven-surefire-reporting-plugin to move the xml results
> parser into the common module
> 
> 5. Refactor Failsafe's verify mojo to use the xml results parse from
> the common module.
> 
> OK, so that's a tad long, but what do people think?
> 
> Any objections?
> 
> -Stephen
> 
> Lazy consensus, 72hrs, i.e. you have until 18:00GMT on 7th Jan 2010 to
> -1 this (after which you'll have to -1 the commit if you feel strongly
> against the direction I'm suggesting)
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
> 
> 


-- 
Dennis Lundberg

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Mime
View raw message