lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dawid Weiss (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-4451) Memory leak per unique thread caused by RandomizedContext.contexts static map
Date Mon, 01 Oct 2012 08:27:07 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-4451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13466691#comment-13466691
] 

Dawid Weiss commented on LUCENE-4451:
-------------------------------------

I looked at the code and I don't have an easy fix for now yet. The problem is that there are
circular reference needs between Threads, Randoms and the runner so that we can assert that
Random instances issued for a thread are not reused on other threads (or outside of a given
test's lifespan). This leads to a cyclic dependency between Thread->PerThreadContext->AssertingRandom->Thread
so even if you put a weak hash map for Thread->PerThreadContext it'll still keep a hard
reference to the thread it's bound to.

I could make AssertingRandom store a weak/soft reference to the thread it's bound to but I'm
kind of afraid it'll affect the performance.

Could we temporarily make a pool of threads for this test and reuse these? I'll be thinking
of a workaround but it's going to take me some time.



                
> Memory leak per unique thread caused by RandomizedContext.contexts static map
> -----------------------------------------------------------------------------
>
>                 Key: LUCENE-4451
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4451
>             Project: Lucene - Core
>          Issue Type: Bug
>            Reporter: Michael McCandless
>            Assignee: Dawid Weiss
>
> In digging on the hard-to-understand OOMEs with
> TestDirectPostingsFormat ... I found (thank you YourKit) that
> RandomizedContext (in randomizedtesting JAR) seems to be holding onto
> all threads created by the test.  The test does create many very short
> lived threads (testing the thread safety of the postings format), in
> BasePostingsFormatTestCase.testTerms), and somehow these seem to tie
> up a lot (~100 MB) of RAM in RandomizedContext.contexts static map.
> For now I've disabled all thread testing (committed {{false &&}} inside
> {{BPFTC.testTerms}}), but hopefully we can fix the root cause here, eg
> when a thread exits can we clear it from that map?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message