lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dawid Weiss <dawid.we...@cs.put.poznan.pl>
Subject Re: ThreadPool threads leaking to suite scope.
Date Fri, 02 Mar 2012 10:22:39 GMT
This leak is really frequent if you preallocate threads on the
execution service. This fragment:

       r.addReaderClosedListener(new ReaderClosedListener() {
         @Override
         public void onClose(IndexReader reader) {
           System.out.println("S:" + ex);
           shutdownExecutorService(ex);
         }
       });

subscribes to the on-close event. I don't see how these listeners can
be properly called in case of something like
SlowCompositeReaderWrapper -- it accepts listeners and subscribes them
to itself but never does anything in doClose. The listener should be
subscribed to the delegate (I think?) but then the method to add
listeners is final up the class hierarchy.

?

Dawid


On Thu, Mar 1, 2012 at 12:20 PM, Uwe Schindler <uwe@thetaphi.de> wrote:
> Hi,
>
> Yeah it's strange. I checked the code: It either does wrap and runs single-thread searches
or it does *not* wrap and runs several threads. So theoretically it should work correctly...
We have to check, if all IndexReaders are correctly closed and the listeners are called.
>
> Uwe
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>
>> -----Original Message-----
>> From: dawid.weiss@gmail.com [mailto:dawid.weiss@gmail.com] On Behalf Of
>> Dawid Weiss
>> Sent: Thursday, March 01, 2012 12:02 PM
>> To: dev@lucene.apache.org
>> Subject: Re: ThreadPool threads leaking to suite scope.
>>
>> I don't know how to fix it, Uwe, but I know there's definitely something not all
>> right with it because threads just keep accumulating (as new searchers are
>> created).
>>
>> I've pushed a static seed for which this is repeatable; this is a heavily worked-on
>> branch but it may lead you to how to fix this:
>>
>> git clone git://github.com/dweiss/lucene_solr.git
>> git checkout 935e1e9e9a350d6b35b23c4545caf78e82b42747
>>
>> try to run TestPhraseQuery (you'll need -ea in Eclipse).
>>
>> Dawid
>>
>> On Thu, Mar 1, 2012 at 11:55 AM, Uwe Schindler <uwe@thetaphi.de> wrote:
>> > I think the problem in newSearcher ist hat sometimes the reader is wrapped.
>> If its wrapped, the underlying reader is only closed, not the wrapper. But the
>> listener is added to the wrapper. We should add the listener to the original
>> inner reader.
>> >
>> > Uwe
>> >
>> > -----
>> > Uwe Schindler
>> > H.-H.-Meier-Allee 63, D-28213 Bremen
>> > http://www.thetaphi.de
>> > eMail: uwe@thetaphi.de
>> >
>> >
>> >> -----Original Message-----
>> >> From: Dawid Weiss [mailto:dawid.weiss@gmail.com]
>> >> Sent: Thursday, March 01, 2012 11:51 AM
>> >> To: dev@lucene.apache.org
>> >> Subject: Re: ThreadPool threads leaking to suite scope.
>> >>
>> >> > 1) initialize threads eagerly; use ThreadPoolExecutor and call
>> >> > prestartAllCoreThreads. this could be applied to LTC on the trunk.
>> >>
>> >> I did this but threads still leak out from unclosed readers created
>> >> by LTC#newSearcher. I don't know why, but this isn't called --
>> >>
>> >>        r.addReaderClosedListener(new ReaderClosedListener() {
>> >>          @Override
>> >>          public void onClose(IndexReader reader) {
>> >>            shutdownExecutorService(ex);
>> >>          }
>> >>        });
>> >>
>> >> Clues?
>> >>
>> >> Dawid
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
>> >> additional commands, e-mail: dev-help@lucene.apache.org
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
>> > additional commands, e-mail: dev-help@lucene.apache.org
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional
>> commands, e-mail: dev-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message