lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: File Handle Leaks During Lucene 3.0.2 Merge
Date Fri, 01 Oct 2010 09:48:02 GMT
On Fri, Oct 1, 2010 at 1:41 AM, Jamie <jamie@stimulussoft.com> wrote:

>  I managed to get hold of a copy of your book through Safari Books. Quite an
> impressive online reading system they have there!

Neat!  I had never heard of Safari Books until now, but it looks cool.

> I integrated your
> SearchManager class into our code, but I am still seeing file handles marked
> deleted in the index directory. I am running the following command on Linux:
>
> sudo watch -n 0 "lsof | grep /var/index | grep deleted | wc -l"
>
> Every 0.1s: lsof | grep /var/index | grep deleted |...  Fri Oct  1 09:37:36
> 2010
>
> 54
>
> The deleted file handles fluctuate up and down. 54 -> 102 -> 64 -> 32, etc.
> They seem stable though. Is this to be expected when using NRT search?
>
>  I am pretty certain that all Searchers are released at the end of every
> search. I double checked it at least twenty times.

SearcherManager always keeps one IndexReader open (the current "live"
one), so it's expected that this reader is holding open references to
deleted files if in fact IndexWriter has completed merges since the
reader was opened. Though.. I would expect you to sometimes see 0
deleted files in your lsof...

Maybe try this: 1) pause your indexing, 2) reopen the reader (call
SearcherManager.reopen), and then 3) run lsof again, at which point
you should see no deleted files still held open.

But note that to truly pause your indexing, you should switch
(temporarily, for this test) your IndexWriter to the
SerialMergeScheduler, and then ensure you stop calling IndexWriter ops
while you run steps 2 and 3.

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message