maven-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sebastian Kirsch (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SUREFIRE-1373) Tests annotated with @NotThreadSafe are not executed in isolation
Date Mon, 15 May 2017 08:25:04 GMT

    [ https://issues.apache.org/jira/browse/SUREFIRE-1373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16010150#comment-16010150
] 

Sebastian Kirsch commented on SUREFIRE-1373:
--------------------------------------------

You're correct, we're using {{parallel}} in order to speed up execution. And yes, there's
a bunch of tests that are considered to be _not thread-safe_, hence we're using the {{@NotThreadSafe}}
annotation to mark them. We assume that those marked tests are executed sequentially in the
specific _non-thread-safe_ thread; though that's not the point. Our assumption is that by
configuring a {{threadCount}} of two, that execution behaves like this:
{noformat}
## 1. parallel execution #########  |
                                    |## non thread-safe execution ##|DONE
## 2. parallel execution ###########|
{noformat}
i.e. there is the configured number of threads running tests in parallel and only afterwards,
the tests marked as non-thread-safe are executed, as suggested by the documentation:
bq. The Thread has name maven-surefire-plugin@NotThreadSafe and it is executed *at the end
of the test run*.

Now I understand that tests are executed like this by design:
{noformat}
## 1. parallel execution #########  |
## non thread-safe execution ##     |DONE
## 2. parallel execution ###########|
{noformat}
If that is case, let me share our use case/understanding of the situation: we have a handful
of tests we consider to be non thread-safe, because they cannot be run in parallel to most
other tests; not only because those marked tests cannot interfere with each other - they manipulate
Environment/System properties to test bootstrapping. If _non-thread-safe_ test execution would
run like I illustrated in the first diagram, we would not run into occasional test execution
failures.

> Tests annotated with @NotThreadSafe are not executed in isolation
> -----------------------------------------------------------------
>
>                 Key: SUREFIRE-1373
>                 URL: https://issues.apache.org/jira/browse/SUREFIRE-1373
>             Project: Maven Surefire
>          Issue Type: Bug
>          Components: Maven Surefire Plugin
>    Affects Versions: 2.20
>            Reporter: Sebastian Kirsch
>            Assignee: Tibor Digana
>
> The [surefire documentation|http://maven.apache.org/components/surefire/maven-failsafe-plugin/examples/fork-options-and-parallel-execution.html]
states that 
> bq. you can apply the JCIP annotation @net.jcip.annotations.NotThreadSafe on the Java
class of JUnit test (pure test class, Suite, Parameterized, etc.) in order to execute it in
single Thread instance. The Thread has name maven-surefire-plugin@NotThreadSafe and it is
executed at the end of the test run.
> However, the thread is run in parallel to the other tests being executed, which contradicts
the documentation and defeats the purpose of running tests sequentially in a single thread.
> See https://github.com/sebastiankirsch/surefire-nonthreadsafe for a test case.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message