maven-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andreas Gudian (JIRA)" <>
Subject [jira] [Commented] (SUREFIRE-1169) JUnit / Arquillian lifecycle friendly test execution with forkCount > 1 and reuseForks = true
Date Sat, 18 Jul 2015 10:30:04 GMT


Andreas Gudian commented on SUREFIRE-1169:

I don't think we should temper with the junit listener-execution from within surefire.
I guess the proposed fix is one way how Falco can have his problem solved. But personally,
I'd just change the Arquillian runner... ;-). I did something similar based on the Spring
runner: just keep the server running until all tests are done, or a test requires a different
That way you have to boot up the server only when necessary and you still get the balancing
that we have right now.


> JUnit / Arquillian lifecycle friendly test execution with forkCount > 1 and reuseForks
= true
> ---------------------------------------------------------------------------------------------
>                 Key: SUREFIRE-1169
>                 URL:
>             Project: Maven Surefire
>          Issue Type: Improvement
>          Components: process forking
>    Affects Versions: 2.18.1
>            Reporter: Falko Modler
>         Attachments: SUREFIRE-1169_forksExecuteTestsOneByOne.patch
> The current approach to "stream" each test class name to a fork via {{TestProvidingInputStream}}
yields a good "load balancing" accross the forks but it *triggers the entire test lifecycle
for each test*.
> With {{forkCount = 1}}, all tests are executed in one set but with {{forkCount = n}}
(n > 1) each test is a separate "set" (so to say).
> This is very problematic in case you or a test framwork you are using relies on a certain
> [Arquillian|] for example ties various "events" to JUnit's lifecycle
methods, like {{AfterSuite}} to {{RunListener.testRunFinished(...)}} which triggers the shutdown
of the managed server etc.
> When using multiple forks, {{RunListener.testRunFinished(...)}} is called for *every*
single test class, triggering {{AfterSuite}} for every single test, although the fork will
receive further tests after that which should "reuse" the server.
> This is just an example. In fact the whole JUnit / Arquillian lifecycle is inconsistent
when using multiple forks.
> From a user perspective I wouldn't expect this behaviour:
> As {{forkCount = 1}} (and {{reusableForks = true}}) triggers {{RunListener.testRunFinished(...)}}
*once*, {{forkCount = n}} (and {{reusableForks = true}}) should trigger that method *n* times,
not *x* times.
> To be fair, the [documentation|]
*does* contain a pointer to that problem by saying:
> {quote}
> When using reuseForks=true and a forkCount value larger than one, test classes are handed
over to the forked process one-by-one.
> {quote}
> But the consequences remain very unclear.
> *(Possible) Solution:*
> I took a stab at this and implemented an "eager test distribution" to the forks in {{}}
and disabled streaming. Please see attached patch (to be applied against project root, 2.18.1).
> Patch Details:
> - New config property: {{ForkConfiguration.forksExecuteTestsOneByOne}}, set via Mojo
(default is true for backward compatibility, name is debatable)
> - When {{forksExecuteTestsOneByOne}} ist set to false, the {{messageQueue}} in {{ForkStarter.runSuitesForkOnceMultiple(...)}}
is *not* wrapped in fork specific {{TestProvidingInputStream}} instances to be consumed bit
by bit later on.
> Instead, the queue is consumed directly and each test class name is assigned to the respective
fork by creating a copy of the {{providerProperties}} which is filled individually for each
> E.g. for three forks and eight tests:
> -- fork 1 executes test 1, 4 and 7
> -- fork 2 executes test 2, 5 and 8
> -- fork 3 executes test 3 and 6
> - To have a clean {{providerProperties}} template I had to move {{DefaultScanResult.writeTo(...)}}
to the respective private methods. Otherwise the properties would have contained *all* tests
> - I refactored some methods in {{ForkStarter}} to enhance readability and to reduce code
> - The patch does *not* contain a test for the new behaviour but all existing tests passed.
> {{forksExecuteTestsOneByOne = false}} now leads to a consistent lifecycle.
> This solution has one downside: One or more forks could be overloaded while other forks
could "underloaded" because you cannot say how long each test will take. runOrder=balanced
could help here but has yet to be implemented for forks.

This message was sent by Atlassian JIRA

View raw message