lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hoss Man (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-3667) Consider changing how we set the number of threads to use to run tests.
Date Fri, 30 Dec 2011 23:14:30 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-3667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13177816#comment-13177816
] 

Hoss Man commented on LUCENE-3667:
----------------------------------

Some  anecdotal numbers on my ThinkPad T400 ("Intel(R) Core(TM)2 Duo CPU     P8800  @ 2.66GHz"
with 8GB RAM running "Linux bester 2.6.31-23-generic #75-Ubuntu SMP Fri Mar 18 18:16:06 UTC
2011 x86_64 GNU/Linux")

"time ant clean compile test" on my laptop this morning, using trunk r1225376...

{noformat}
BUILD SUCCESSFUL
Total time: 32 minutes 27 seconds

real	32m28.495s
user	33m12.050s
sys	1m47.760s
{noformat}

...during that run, my CPU temp monitor warned several times (4 i think? it's a notification
bar thing, i don't believe i have a log of it) that my CPUs were spiking into the 70-75C range.

same command after svn updating to r1225945 ....

{noformat}
BUILD SUCCESSFUL
Total time: 18 minutes 49 seconds

real	18m50.329s
user	23m44.440s
sys	1m1.080s
{noformat}

...and my CPU only spiked up to the ~70C range once.

+1 ... thanks rmuir.
                
> Consider changing how we set the number of threads to use to run tests.
> -----------------------------------------------------------------------
>
>                 Key: LUCENE-3667
>                 URL: https://issues.apache.org/jira/browse/LUCENE-3667
>             Project: Lucene - Java
>          Issue Type: Improvement
>            Reporter: Mark Miller
>            Assignee: Mark Miller
>            Priority: Minor
>         Attachments: LUCENE-3667.patch, LUCENE-3667.patch, LUCENE-3667.patch, LUCENE-3667.patch,
LUCENE-3667.patch
>
>
> The current way we set the number of threads to use is not expressive enough for some
systems. My quad core with hyper threading is recognized as 8 CPUs - since I can only override
the number of threads to use per core, 8 is as low as I can go. 8 threads can be problematic
for me - just the amount of RAM used sometimes can toss me into heavy paging because I only
have 8 GB of RAM - the heavy paging can cause my whole system to come to a crawl. Without
hacking the build, I don't think I have a lot of workarounds.
> I'd like to propose that switch from using threadsPerProcessor to threadCount. In some
ways, it's not as nice, because it does not try to scale automatically per system. But that
auto scaling is often not ideal (hyper threading, wanting to be able to do other work at the
same time), so perhaps we just default to 1 or 2 threads and devs can override individually?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message