maven-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "M.P. Korstanje (Jira)" <>
Subject [jira] [Created] (SUREFIRE-1691) surefire-junit47 reports results against incorrect test
Date Sun, 01 Sep 2019 12:37:00 GMT
M.P. Korstanje created SUREFIRE-1691:

             Summary: surefire-junit47 reports results against incorrect test
                 Key: SUREFIRE-1691
             Project: Maven Surefire
          Issue Type: Bug
          Components: Junit 4.x support
    Affects Versions: 3.0.0-M3
            Reporter: M.P. Korstanje

This is a follow up on:
 - []

The latest reproducer can be found here:
 - []

In short Cucumber has a hierarchy of tests that looks at follows:
Class annotated with @RunWith(Cucumber.class)
 |- Feature File 1
 |  |- Scenario 1a
 |  |- Scenario 1b
 |- Feature File 2
 |  |- Scenario 2a
 |  |- Scenario 2b
Surefire tries to group output in tests sets, either per class or per test suite. As Cucumber
currently only emits test started/finished events for Scenarios sure fire can not group scenarios
into a test set. They are instead group with the previous test set. However even after adding
test started/finished events for features results in a single test set.

I believe there is an error in {{NonConcurrentRunListener.describesNewTestSet}}.
    private boolean describesNewTestSet( Description description )
        if ( currentTestSetDescription != null )
            if ( null != description.getTestClass() )
                return !description.getTestClass().equals( currentTestSetDescription.getTestClass()
            else if ( description.isSuite() )
                return description.getChildren().equals( currentTestSetDescription.getChildren()

            return false;

        return true;
The value of {{description.getChildren().equals( currentTestSetDescription.getChildren() );}}
should be negated so so different test sets are properly recognized.

Fixing this locally works and can be reproduced by:

1. Negating the line and building sure fire locally
 2. Building [] with `mvn clean install
 3. Running `mvn clean test -Pjunit47` on []

It does however does result in another unwanted result: A feature file with two scenarios
is now reported as having 3 results. One for each scenario and one for the feature.
Test set: Some Feature 1
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.112 s - in Some_Feature_1

This message was sent by Atlassian Jira

View raw message