Return-Path: 
 <users-return-128626-apmail-maven-users-archive=maven.apache.org@maven.apache.org>
X-Original-To: apmail-maven-users-archive@www.apache.org
Delivered-To: apmail-maven-users-archive@www.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id D3E52DE86
	for <apmail-maven-users-archive@www.apache.org>;
 Thu,  9 Aug 2012 14:23:45 +0000 (UTC)
Received: (qmail 21063 invoked by uid 500); 9 Aug 2012 14:23:43 -0000
Delivered-To: apmail-maven-users-archive@maven.apache.org
Received: (qmail 21001 invoked by uid 500); 9 Aug 2012 14:23:43 -0000
Mailing-List: contact users-help@maven.apache.org; run by ezmlm
Precedence: bulk
List-Unsubscribe: <mailto:users-unsubscribe@maven.apache.org>
List-Help: <mailto:users-help@maven.apache.org>
List-Post: <mailto:users@maven.apache.org>
List-Id: "Maven Users List" <users.maven.apache.org>
Reply-To: "Maven Users List" <users@maven.apache.org>
Delivered-To: mailing list users@maven.apache.org
Received: (qmail 20993 invoked by uid 99); 9 Aug 2012 14:23:43 -0000
Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 09 Aug 2012 14:23:43 +0000
X-ASF-Spam-Status: No, hits=1.5 required=5.0
	tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: domain of stephen.alan.connolly@gmail.com
 designates 74.125.82.41 as permitted sender)
Received: from [74.125.82.41] (HELO mail-wg0-f41.google.com) (74.125.82.41)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 09 Aug 2012 14:23:35 +0000
Received: by wgbds1 with SMTP id ds1so351574wgb.0
        for <users@maven.apache.org>; Thu, 09 Aug 2012 07:23:15 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20120113;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        bh=pGTcnIcNArCijScLRQROiP46i8y4PxrZGXwMjvcbOUE=;
        b=sLwYbRogMJ44sUR4kf4+BHlXORsEYZfisvuX1kbKvL1v81JxUToRKYyDIb+lT28PLZ
         HDEpt7z0dTppomptBQl3JiA1hTJv6agsECj4vJbr5TJ+wjC8ms89q/jER7+y3VnBMEL7
         1q95abaqS4uTBu0bujBW/hyyREpLcz6aIxBA7KcJCS27QgHD8RSFkWXN+sIaASC81OAV
         JYuPb6Z8TyW4OPb3CnOaU2o4rv53CQ+f9Az4y6Y6lAp4FZ+0TA5N5YeRJQLkVEMdR4PX
         Xh8MCByJXpqK/g1VczXieot9CA7T4hmNv0pKiHQR2s/Ge+XvKljUmw8N29+wtjFUQ83C
         NaSA==
MIME-Version: 1.0
Received: by 10.180.104.200 with SMTP id gg8mr3328020wib.14.1344522194965;
 Thu, 09 Aug 2012 07:23:14 -0700 (PDT)
Received: by 10.217.0.4 with HTTP; Thu, 9 Aug 2012 07:23:14 -0700 (PDT)
In-Reply-To: 
 <CAJSLXwLE8bVvUGqPJAcak3NDxvMtFZmNxkQJpwgoOpP4aEVHeA@mail.gmail.com>
References: 
 <CAJSLXwK5Uc7mQOhFex2dn2sh=Mjsgr-RuAbZ2f5zfjEyQqeE9g@mail.gmail.com>
	<CAJSLXwLE8bVvUGqPJAcak3NDxvMtFZmNxkQJpwgoOpP4aEVHeA@mail.gmail.com>
Date: Thu, 9 Aug 2012 15:23:14 +0100
Message-ID: 
 <CA+nPnMzR3GxfkbRHxwGmRKtiBQbjty=pVYi6MS-rE+t6iu17fA@mail.gmail.com>
Subject: Re: failsafe report displays no results
From: Stephen Connolly <stephen.alan.connolly@gmail.com>
To: Maven Users List <users@maven.apache.org>
Content-Type: multipart/alternative; boundary=f46d04430404d2d51b04c6d5f7da

--f46d04430404d2d51b04c6d5f7da
Content-Type: text/plain; charset=ISO-8859-1

On 27 July 2012 01:59, John Rodriguez <john.rodriguez@gmail.com> wrote:

> I just started using "maven-failsafe-plugin" and
> "maven-surefire-report-plugin" so my apologies in advance...
>
> For starters, I've configured maven-failsafe-plugin and
> maven-surefire-report-plugin as such (respectively):
>
>     <build>
>         <plugins>
>             <plugin>
>                 <groupId>org.apache.maven.plugins</groupId>
>                 <artifactId>maven-failsafe-plugin</artifactId>
>                 <version>2.12</version>
>                 <executions>
>                     <execution>
>                         <goals>
>                             <goal>integration-test</goal>
>                             <goal>verify</goal>
>                         </goals>
>                     </execution>
>                 </executions>
>             </plugin>
> ...
>     <reporting>
>         <plugins>
>               <plugin>
>                 <groupId>org.apache.maven.plugins</groupId>
>                 <artifactId>maven-surefire-report-plugin</artifactId>
>                 <version>2.12</version>
>                 <reportSets>
>                     <reportSet>
>                         <id>integration-tests</id>
>                         <reports>
>                             <report>failsafe-report-only</report>
>                         </reports>
>                     </reportSet>
>                 </reportSets>
>               </plugin>
>         </plugins>
>     </reporting>
> ...
>
> I have two test cases defined in /src/test/java/.../GameResourceIT.java
> like so:
>
> public class GameResourceIT extends TestCase {
>     public static final String LOCAL_SERVER_URL_ROOT =
> "http://localhost:8080/gameservice";
>     public static final String GAMES_RESOURCE_URI = "/games";
>
>     public void testGetGamesForDate_20120603_status200Expected()
> throws Exception {
>         String date = "06-03-2012";
>         Client client = Client.create();
>         WebResource webResource =
> client.resource(LOCAL_SERVER_URL_ROOT + GAMES_RESOURCE_URI + "/" +
> date);
>         ClientResponse response =
> webResource.accept("application/json").get(ClientResponse.class);
>         assertEquals("Failed : status = " + response.getStatus() + ",
> body = " + response.getEntity(String.class),
>                 response.getStatus(), Status.OK.getStatusCode());
>     }
>
>     public void testGetGamesForDate_20120608_status404Expected()
> throws Exception {
>         String date = "06-08-2012";
>         Client client = Client.create();
>         WebResource webResource =
> client.resource(LOCAL_SERVER_URL_ROOT + GAMES_RESOURCE_URI + "/" +
> date);
>         ClientResponse response =
> webResource.accept("application/json").get(ClientResponse.class);
>         assertEquals("Failed : status = " + response.getStatus() + ",
> body = " + response.getEntity(String.class),
>                 response.getStatus(), Status.NOT_FOUND.getStatusCode());
>     }
> }
>
> ...
>
> When I run "mvn verify", I get the following:
>
> [INFO] --- maven-failsafe-plugin:2.12:integration-test (default) @
> gameservice ---
> [INFO] Failsafe report directory: ...\gameservice\target\failsafe-reports
> -------------------------------------------------------
>  T E S T S
> -------------------------------------------------------
> Running ....GameResourceIT
> Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.938
> sec <<< FAILURE!
>
> Results :
> Failed tests:
> testGetGamesForDate_20120608_status404Expected(...GameResourceIT):
> Failed : status = 200, body = null expected:<200> but was:<404>
> Tests run: 2, Failures: 1, Errors: 0, Skipped: 0
>
> [INFO] --- maven-failsafe-plugin:2.12:verify (default) @ gameservice ---
> [INFO] Failsafe report directory:...\gameservice\target\failsafe-reports
> [INFO]
> ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 13.753s
> [INFO] Finished at: Thu Jul 26 20:39:44 EDT 2012
> 2012-07-26 20:39:44.644::INFO:  Shutdown hook complete
> [INFO] Final Memory: 21M/52M
> [INFO]
> ------------------------------------------------------------------------
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-failsafe-plugin:2.12:verify (default)
> on project gameservice: There are test failures.
>
>
> Fine.  As expected, 1 of the 2 tests failed.  But as NOT expected, the
> verify goal failed as well (I thought integration test failures would
> not halt execution, but simply be recorded).
>

the failsafe:integration-test goal will never fail the build.... failing
the build is the job of the failsafe:verify goal.

thus when bound to the lifecycle at the appropriate phases the plugins
bound to the post-integration-test phase will execute.

Could you please suggest how we could clarify the documentation for the
plugin to make this clearer since it obviously was misinterpreted by you.


>
> Now, if I look at /target/failsafe-reports/GameResourceIT.txt and
> /target/failsafe-reports/TEST-GameResourceIT.xml, I see as expected
> that 1 of 2 tests failed.
> When I look at /target/failsafe-reports/failsafe-summary.xml, I see:
> <failsafe-summary result="255" />
>
> Finally, if I run "mvn:site surefire-report:report", I get an html
> page in /target/site/surefire-report.html that shows all 0s for the
> report summary (tests, errors, failures, skipped, etc.)
>
> Why would this be the case?  I expected maven-surefire-report-plugin
> to transform the file(s) located in /target/failsafe-reports into an
> equivalent HTML report (indicating that 1 of 2 tests failed) located
> in /target/site.
>

IIRC there is a separate report for generating the failsafe reports, and
additionally it will be empty if you have run mvn clean in between running
mvn verify and running mvn site as unlike the surefire report it does not
fork a lifecycle (since integration tests are in general more brittle and
longer running and forking a lifecycle all the way up to the verify phase
would cause a world of pain to end users.


>
> Is my configuration off?
>

Yes, but please tell us how to improve our documentation so that others do
not get as confused as you have.


>
> -JR
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>

--f46d04430404d2d51b04c6d5f7da--