maven-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tibor Digana (JIRA)" <j...@codehaus.org>
Subject [jira] (SUREFIRE-1048) Surefire does not use the total amount of threads specified by threadCount
Date Sat, 04 Jan 2014 09:20:45 GMT

    [ https://jira.codehaus.org/browse/SUREFIRE-1048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=338071#comment-338071
] 

Tibor Digana commented on SUREFIRE-1048:
----------------------------------------

Yes it can be closed.
The user should hava a chance to change the performance of the build using these params. The
same params are used to express that e.g. methods do not have any explicitly defined thread-count.
For instance in version 2.16 if parallel=all, threadCount=16 and threadCountSuites=2, threadCountClasses=5,
where threadCountMethods is left unspecified, then concurrency of methods is varying between
9 - 14 or 15. The same may happen with classes if parallel=suitesAndClasses in 2.17.
If you specify the ration e.g. 10%-20%-70% per every entity of param "parallel" then the only
parameter which is mostly used in performance tuning is threadCount.
There are two new parameters of timeout in this parallelism.
                
> Surefire does not use the total amount of threads specified by threadCount
> --------------------------------------------------------------------------
>
>                 Key: SUREFIRE-1048
>                 URL: https://jira.codehaus.org/browse/SUREFIRE-1048
>             Project: Maven Surefire
>          Issue Type: Bug
>          Components: Junit 4.7+ (parallel) support
>    Affects Versions: 2.16
>            Reporter: Fabio Kenji Kume
>         Attachments: surefire-parallel-test.zip
>
>
> Hello,
> We've been using Surefire Plugin's parallel settings since 2.16 in order to run our selenium-based
integration tests.
> The way we run it is by using the following parameters:
> parallel='all', perCoreThreadCount=false, threadCount=20, useUnlimitedThreads=false,
threadCountSuites=0, threadCountClasses=0, threadCountMethods=0
> We are running a Junit Suite using Categories. What we've noticed, though, is that even
if we specify the total thread count as 20, we never see that number of tests being run in
parallel. We could check that fact by viewing our selenium-grid's console, where we would
see that the number of selenium grid nodes that were being used for tests were never the full
20.
> We've played around with different threadCount numbers and got the following results:
> threadCount / slots being used at selenium-grid
> 3  / 1
> 4  / 2
> 5  / 3
> 6  / 3
> 7  / 4
> 8  / 5
> 9  / 5
> 10 / 6
> 11 / 7
> 13 / 8
> It seems the number of threads being used is never the total of threads specified. 
> I've created another project so I could test this around, which is attached to this issue.
> In it, I've created a Junit rule that will print the whenever a test starts, and a Test
Suite using categories from another testcase. 
> When I run it using threadCount = 10, I get the following:
> mvn clean test -Dtest=**/ExpSuite.java
> reports
> [INFO] parallel='all', perCoreThreadCount=false, threadCount=10, useUnlimitedThreads=false,
threadCountSuites=0, threadCountClasses=0, threadCountMethods=0
> -------------------------------------------------------
>  T E S T S
> -------------------------------------------------------
> Running second.ExpTest
> [Mon Dec 23 16:11:15 BRST 2013] START test_1
> [Mon Dec 23 16:11:15 BRST 2013] START test_7
> [Mon Dec 23 16:11:15 BRST 2013] START test_6
> [Mon Dec 23 16:11:15 BRST 2013] START test_3
> [Mon Dec 23 16:11:15 BRST 2013] START test_5
> [Mon Dec 23 16:11:15 BRST 2013] START test_10
> [Mon Dec 23 16:11:15 BRST 2013] START test_4
> [Mon Dec 23 16:11:15 BRST 2013] START test_2
> [Mon Dec 23 16:15:30 BRST 2013] START test_8
> [Mon Dec 23 16:15:32 BRST 2013] START test_9
> Seems that last two tests didn't get to run at the same time as the others, since they
have very different date values. However, if I run them specifying the exact number of threadCountSuites,
threadCountClasses and threadCountMethods, the tests run all in parallel:
> [INFO] parallel='all', perCoreThreadCount=false, threadCount=0, useUnlimitedThreads=false,
threadCountSuites=10, threadCountClasses=10, threadCountMethods=10
> -------------------------------------------------------
>  T E S T S
> -------------------------------------------------------
> Running second.ExpTest
> [Mon Dec 23 15:50:51 BRST 2013] START test_2
> [Mon Dec 23 15:50:51 BRST 2013] START test_7
> [Mon Dec 23 15:50:51 BRST 2013] START test_8
> [Mon Dec 23 15:50:51 BRST 2013] START test_9
> [Mon Dec 23 15:50:51 BRST 2013] START test_10
> [Mon Dec 23 15:50:51 BRST 2013] START test_1
> [Mon Dec 23 15:50:51 BRST 2013] START test_3
> [Mon Dec 23 15:50:51 BRST 2013] START test_4
> [Mon Dec 23 15:50:51 BRST 2013] START test_6
> [Mon Dec 23 15:50:51 BRST 2013] START test_5
> Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3,320.712 sec - in second.ExpTest
> If I try running only the test class (ExpTest), I get the following:
> mvn clean test -Dtest=**/ExpTest.java
> [INFO] parallel='all', perCoreThreadCount=false, threadCount=10, useUnlimitedThreads=false,
threadCountSuites=0, threadCountClasses=0, threadCountMethods=0
> -------------------------------------------------------
>  T E S T S
> -------------------------------------------------------
> Running second.ExpTest
> [Mon Dec 23 17:07:46 BRST 2013] START test_3
> [Mon Dec 23 17:07:46 BRST 2013] START test_8
> [Mon Dec 23 17:07:46 BRST 2013] START test_1
> [Mon Dec 23 17:07:46 BRST 2013] START test_2
> [Mon Dec 23 17:07:46 BRST 2013] START test_7
> [Mon Dec 23 17:07:46 BRST 2013] START test_6
> [Mon Dec 23 17:07:46 BRST 2013] START test_10
> [Mon Dec 23 17:07:46 BRST 2013] START test_5
> [Mon Dec 23 17:07:46 BRST 2013] START test_4
> [Mon Dec 23 17:12:37 BRST 2013] START test_9
> Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2,710.144 sec - in second.ExpTest
> We still didn't get the 10 threads to run.
> I've looked around the source code and it seems to be related to how surefire calculates
the ratio of threads that should be allocated to run methods, suites and classes.
> Could you help me with this? 
> Thank you,
> Fabio Kenji

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message