lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Justin <cry...@yahoo.com>
Subject Re: File descriptor leak in ParallelReader.reopen()
Date Mon, 08 Mar 2010 19:52:14 GMT
So I learned that IndexReader objects must be closed immediately following a reopen.  But what
is the guidance after using IndexWriter.getReader()?  Does one reader need closed before getting
and using another, or can one use 2 or more readers simultaneously?  I don't see any guidance
at the following references, although there is mention that reopen() will forward back to
getReader().

http://lucene.apache.org/java/3_0_1/api/core/org/apache/lucene/index/IndexWriter.html#getReader%28%29

http://wiki.apache.org/lucene-java/NearRealtimeSearch




----- Original Message ----
From: Justin <crynax@yahoo.com>
To: java-user@lucene.apache.org
Sent: Fri, March 5, 2010 10:24:20 AM
Subject: Re: File descriptor leak in ParallelReader.reopen()

Out of curiosity, I downloaded 2.4.1 and made the necessary source code modifications (attached).
 There used to be some sort of file descriptor cleanup.  With the explicit close, the descriptor
count stayed under 100.  Without the explicit close, the count hit peaks around 3000 by the
time 50,000 documents are added (still under our increased limit).  Hopefully our problems
didn't extend past leaking file descriptors by omitting the explicit close.




----- Original Message ----
From: Justin <crynax@yahoo.com>
To: java-user@lucene.apache.org
Sent: Thu, March 4, 2010 6:29:25 PM
Subject: Re: File descriptor leak in ParallelReader.reopen()

We must have been getting lucky.  Thanks Mark and Uwe!




----- Original Message ----
From: Uwe Schindler <uwe@thetaphi.de>
To: java-user@lucene.apache.org
Sent: Thu, March 4, 2010 6:20:56 PM
Subject: RE: File descriptor leak in ParallelReader.reopen()

That was always the same with reopen(). Its documented in the javadocs, with a short example:
http://lucene.apache.org/java/3_0_1/api/all/org/apache/lucene/index/IndexReader.html#reopen()

also in 2.4.1:
http://lucene.apache.org/java/2_4_1/api/org/apache/lucene/index/IndexReader.html#reopen()

Uwe

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de


> -----Original Message-----
> From: Justin [mailto:crynax@yahoo.com]
> Sent: Friday, March 05, 2010 1:17 AM
> To: java-user@lucene.apache.org
> Subject: Re: File descriptor leak in ParallelReader.reopen()
> 
> Has this changed since 2.4.1?  Our application didn't explicitly close
> with 2.4.1 and that combination never had this problem.
> 
> 
> 
> ----- Original Message ----
> From: Mark Miller <markrmiller@gmail.com>
> To: java-user@lucene.apache.org
> Sent: Thu, March 4, 2010 6:00:02 PM
> Subject: Re: File descriptor leak in ParallelReader.reopen()
> 
> On 03/04/2010 06:52 PM, Justin wrote:
> > Hi Mike and others,
> >
> > I have a test case for you (attached) that exhibits a file descriptor
> leak in ParallelReader.reopen().  I listed the OS, JDK, and snapshot of
> Lucene that I'm using in the source code.
> >
> > A loop adds just over 4000 documents to an index, reopening the index
> after each, before my system hits an already increased file descriptor
> limit of 8192.  I've also got a thread that reports the number of
> documents in the index and warms a searcher using the reader.  To
> simulate continued use by my application the searchers are not
> discarded.
> >
> > Let me know if you need help reproducing the problem or can help
> identify it.
> >
> > Thanks!
> > Justin
> >
> >
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> Doesn't look like you are closing your old reader - reopen will return
> a
> new one when there are changes to the index and the old one must be
> closed.
> 
> --
> - Mark
> 
> http://www.lucidimagination.com
> 
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


      

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


      
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


      

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message