maven-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "M.P. Korstanje (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SUREFIRE-1372) Rerunning failing tests fails in combination with Description#createSuiteDescription
Date Sat, 27 May 2017 10:42:04 GMT

    [ https://issues.apache.org/jira/browse/SUREFIRE-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16027395#comment-16027395
] 

M.P. Korstanje commented on SUREFIRE-1372:
------------------------------------------

>  Ok, lets try to use latest cucumber release version without having 2.0.0-SNAPSHOT and
run the build of surefire mvn install -P run-its. Is the build successful?

No. {{cucumber-jvm:1.2.5}} does not provided proper id objects in Descriptions. They don't
implement equals and hashcode making it impossible to rerun failing tests. This causes the
the failure of JUnit47RerunFailingTestWithCucumberIT and JUnit4RerunFailingTestWithCucumberIT.

Additionally I'm getting incorrect runcounts for JUnit4RerunFailingTestsIT because it does
not create a new {{Result}} object for the rerun session. This did not bother the old implementation
because by going straight to the execution of the class method a testFinished event was never
fired. Currently considering how to resolve this, but at the moment I reckon it'd be best
to only support the Junit47Provider. Not unreasonable as cucumber requires Junit 4.12.

> Is there any way of using CucumberOptions with re-run in the old release versions of
cucumber?

No. Cucumber.Options was deprecated in favor of CucumberOptions and has been removed in {{cucumber-jvm:1.2.0}}.
This unfortunately did not result in a major release.

> Any drawback?

CucumberOptions does not control or influence the rerun. It is not required. However not using
{{@CucumberOptions(junit = \{"--no-step-notifications"\})}} means cucumber will report cucumber
steps (the line in a test) as tests rather then cucumber scenarios (the actual tests). This
makes the test count and reports look odd. 

As shown in: 
https://github.com/apache/maven-surefire/blob/cba4adb1b93002c5b4bb2d2f22f461cc53bd8738/surefire-integration-tests/src/test/java/org/apache/maven/surefire/its/JUnit47WithCucumberIT.java#L57-L59

> Maybe the users of surefire and Cucubmer would prefer some documentation in surefire
if current CucumberOptions are possible.

Do you have any suggestion where I might submit such documentation?


> Rerunning failing tests fails in combination with Description#createSuiteDescription
> ------------------------------------------------------------------------------------
>
>                 Key: SUREFIRE-1372
>                 URL: https://issues.apache.org/jira/browse/SUREFIRE-1372
>             Project: Maven Surefire
>          Issue Type: Bug
>          Components: Maven Surefire Plugin
>    Affects Versions: 2.20
>            Reporter: M.P. Korstanje
>            Assignee: Tibor Digana
>
> When using surefire to rerun failing tests created by a Runner that uses {noformat}Description#createSuiteDescription{noformat}
with a human readable name rather then a class name the following stack trace occurs:
> {code}
> org.apache.maven.surefire.testset.TestSetFailedException: Unable to create test class
'Scenario: Fail when running'
>         at org.apache.maven.surefire.junit4.JUnit4Provider.executeFailedMethod(JUnit4Provider.java:385)
>         at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:292)
>         at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
>         at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:161)
> Caused by: java.lang.ClassNotFoundException: Scenario: Fail when running
>         at org.apache.maven.surefire.junit4.JUnit4Provider.executeFailedMethod(JUnit4Provider.java:379)
>         at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:292)
>         at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
>         at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:161)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message