lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Muir (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-3667) Consider changing how we set the number of threads to use to run tests.
Date Mon, 02 Jan 2012 17:18:30 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-3667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13178464#comment-13178464
] 

Robert Muir commented on LUCENE-3667:
-------------------------------------

Thanks for reporting back guys. I still dont like the timings hossman has (i think 19 minutes
is crazy, would really love to know whats going on there).

but just for comparison here is my machines:

Linux (i7-2600k@3.4ghz, 8gb ram):

Before:
{noformat}
BUILD SUCCESSFUL
Total time: 7 minutes 2 seconds

real	7m3.099s
user	27m47.900s
sys	0m54.639s
{noformat}

After:
{noformat}
BUILD SUCCESSFUL
Total time: 4 minutes 51 seconds

real	4m52.310s
user	17m14.869s
sys	0m29.682s
{noformat}

Windows (Core2Quad-Q9650@3.0ghz, 8gb ram)

Before:
{noformat}
-Solr tests always timeout/fail-
{noformat}

After:
{noformat}
BUILD SUCCESSFUL
Total time: 8 minutes 37 seconds

real    8m39.302s
user    0m0.000s
sys     0m0.046s
{noformat}

Mac (Core i5@2.3ghz, 4gb ram)

Before:
{noformat}
-Solr tests always timeout/fail-
{noformat}

After:
{noformat}
BUILD SUCCESSFUL
Total time: 11 minutes 20 seconds

real    11m20.428s
user    28m0.921s
sys     1m38.629s
{noformat}
                
> Consider changing how we set the number of threads to use to run tests.
> -----------------------------------------------------------------------
>
>                 Key: LUCENE-3667
>                 URL: https://issues.apache.org/jira/browse/LUCENE-3667
>             Project: Lucene - Java
>          Issue Type: Improvement
>            Reporter: Mark Miller
>            Assignee: Mark Miller
>            Priority: Minor
>         Attachments: LUCENE-3667.patch, LUCENE-3667.patch, LUCENE-3667.patch, LUCENE-3667.patch,
LUCENE-3667.patch
>
>
> The current way we set the number of threads to use is not expressive enough for some
systems. My quad core with hyper threading is recognized as 8 CPUs - since I can only override
the number of threads to use per core, 8 is as low as I can go. 8 threads can be problematic
for me - just the amount of RAM used sometimes can toss me into heavy paging because I only
have 8 GB of RAM - the heavy paging can cause my whole system to come to a crawl. Without
hacking the build, I don't think I have a lot of workarounds.
> I'd like to propose that switch from using threadsPerProcessor to threadCount. In some
ways, it's not as nice, because it does not try to scale automatically per system. But that
auto scaling is often not ideal (hyper threading, wanting to be able to do other work at the
same time), so perhaps we just default to 1 or 2 threads and devs can override individually?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message