maven-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephen Connolly <stephen.alan.conno...@gmail.com>
Subject Re: failsafe report displays no results
Date Thu, 09 Aug 2012 14:23:14 GMT
On 27 July 2012 01:59, John Rodriguez <john.rodriguez@gmail.com> wrote:

> I just started using "maven-failsafe-plugin" and
> "maven-surefire-report-plugin" so my apologies in advance...
>
> For starters, I've configured maven-failsafe-plugin and
> maven-surefire-report-plugin as such (respectively):
>
>     <build>
>         <plugins>
>             <plugin>
>                 <groupId>org.apache.maven.plugins</groupId>
>                 <artifactId>maven-failsafe-plugin</artifactId>
>                 <version>2.12</version>
>                 <executions>
>                     <execution>
>                         <goals>
>                             <goal>integration-test</goal>
>                             <goal>verify</goal>
>                         </goals>
>                     </execution>
>                 </executions>
>             </plugin>
> ...
>     <reporting>
>         <plugins>
>               <plugin>
>                 <groupId>org.apache.maven.plugins</groupId>
>                 <artifactId>maven-surefire-report-plugin</artifactId>
>                 <version>2.12</version>
>                 <reportSets>
>                     <reportSet>
>                         <id>integration-tests</id>
>                         <reports>
>                             <report>failsafe-report-only</report>
>                         </reports>
>                     </reportSet>
>                 </reportSets>
>               </plugin>
>         </plugins>
>     </reporting>
> ...
>
> I have two test cases defined in /src/test/java/.../GameResourceIT.java
> like so:
>
> public class GameResourceIT extends TestCase {
>     public static final String LOCAL_SERVER_URL_ROOT =
> "http://localhost:8080/gameservice";
>     public static final String GAMES_RESOURCE_URI = "/games";
>
>     public void testGetGamesForDate_20120603_status200Expected()
> throws Exception {
>         String date = "06-03-2012";
>         Client client = Client.create();
>         WebResource webResource =
> client.resource(LOCAL_SERVER_URL_ROOT + GAMES_RESOURCE_URI + "/" +
> date);
>         ClientResponse response =
> webResource.accept("application/json").get(ClientResponse.class);
>         assertEquals("Failed : status = " + response.getStatus() + ",
> body = " + response.getEntity(String.class),
>                 response.getStatus(), Status.OK.getStatusCode());
>     }
>
>     public void testGetGamesForDate_20120608_status404Expected()
> throws Exception {
>         String date = "06-08-2012";
>         Client client = Client.create();
>         WebResource webResource =
> client.resource(LOCAL_SERVER_URL_ROOT + GAMES_RESOURCE_URI + "/" +
> date);
>         ClientResponse response =
> webResource.accept("application/json").get(ClientResponse.class);
>         assertEquals("Failed : status = " + response.getStatus() + ",
> body = " + response.getEntity(String.class),
>                 response.getStatus(), Status.NOT_FOUND.getStatusCode());
>     }
> }
>
> ...
>
> When I run "mvn verify", I get the following:
>
> [INFO] --- maven-failsafe-plugin:2.12:integration-test (default) @
> gameservice ---
> [INFO] Failsafe report directory: ...\gameservice\target\failsafe-reports
> -------------------------------------------------------
>  T E S T S
> -------------------------------------------------------
> Running ....GameResourceIT
> Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.938
> sec <<< FAILURE!
>
> Results :
> Failed tests:
> testGetGamesForDate_20120608_status404Expected(...GameResourceIT):
> Failed : status = 200, body = null expected:<200> but was:<404>
> Tests run: 2, Failures: 1, Errors: 0, Skipped: 0
>
> [INFO] --- maven-failsafe-plugin:2.12:verify (default) @ gameservice ---
> [INFO] Failsafe report directory:...\gameservice\target\failsafe-reports
> [INFO]
> ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 13.753s
> [INFO] Finished at: Thu Jul 26 20:39:44 EDT 2012
> 2012-07-26 20:39:44.644::INFO:  Shutdown hook complete
> [INFO] Final Memory: 21M/52M
> [INFO]
> ------------------------------------------------------------------------
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-failsafe-plugin:2.12:verify (default)
> on project gameservice: There are test failures.
>
>
> Fine.  As expected, 1 of the 2 tests failed.  But as NOT expected, the
> verify goal failed as well (I thought integration test failures would
> not halt execution, but simply be recorded).
>

the failsafe:integration-test goal will never fail the build.... failing
the build is the job of the failsafe:verify goal.

thus when bound to the lifecycle at the appropriate phases the plugins
bound to the post-integration-test phase will execute.

Could you please suggest how we could clarify the documentation for the
plugin to make this clearer since it obviously was misinterpreted by you.


>
> Now, if I look at /target/failsafe-reports/GameResourceIT.txt and
> /target/failsafe-reports/TEST-GameResourceIT.xml, I see as expected
> that 1 of 2 tests failed.
> When I look at /target/failsafe-reports/failsafe-summary.xml, I see:
> <failsafe-summary result="255" />
>
> Finally, if I run "mvn:site surefire-report:report", I get an html
> page in /target/site/surefire-report.html that shows all 0s for the
> report summary (tests, errors, failures, skipped, etc.)
>
> Why would this be the case?  I expected maven-surefire-report-plugin
> to transform the file(s) located in /target/failsafe-reports into an
> equivalent HTML report (indicating that 1 of 2 tests failed) located
> in /target/site.
>

IIRC there is a separate report for generating the failsafe reports, and
additionally it will be empty if you have run mvn clean in between running
mvn verify and running mvn site as unlike the surefire report it does not
fork a lifecycle (since integration tests are in general more brittle and
longer running and forking a lifecycle all the way up to the verify phase
would cause a world of pain to end users.


>
> Is my configuration off?
>

Yes, but please tell us how to improve our documentation so that others do
not get as confused as you have.


>
> -JR
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message