lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: File Handle Leaks During Lucene 3.0.2 Merge
Date Thu, 30 Sep 2010 21:56:19 GMT
On Thu, Sep 30, 2010 at 5:59 AM, Jamie <jamie@stimulussoft.com> wrote:
>  Hi Michael / Uwe
>
>>It's good to cache the reader, but, finalize would worry me too since
>>you have no control over when GC gets around to calling it... you risk
>>tying up resources for longer than necessary.
>
> I did it this way, as I didn't want to over complicate the code by
> introducing mechanisms to track the number of search threads using a shared
> indexreader. Admittedly, its not a very clean solution but in my case it
> does work. Is there a particular technique for knowing when to a close a
> reader when there are multiple search threads using that reader? Should I
> keep some kind of counter and override the close method of the reader such
> that the underlying reader is only closed when everyone's done with it?

See Uwe's response (or SearcherManager).

>> IndexWriter has a reader pool, internally, where it holds open
>> SegmentReaders for the still-live segments in the index.  This is used
>> by IndexReader.reopen to share open SegmentReaders.
>>
>> But the open files should correspond only to segments still "live" in
>> the index.  After segments are merged away, these readers are dropped.
>>  Is this what you are seeing?
>>
> I dont fully understand your explanation/question. When I run lsof, I am
> seeing the following:
>
> /usr/local/mailarchiva/server/webapps/ROOT/WEB-INF/logs/index/_jyr.cfs
> (deleted)
> /usr/local/mailarchiva/server/webapps/ROOT/WEB-INF/logs/index/_jyp.cfs
> (deleted)
>
> I assume these are left by the OS after the merge operation tried to delete
> old segments. The OS is unable to delete the files. I think its because our
> new code never closes the indexwriter, but rather uses the
> indexwriter.commit() method to apply the changes. Is this correct?

Ahh I see they are deleted but held open... hmmm.

Though this is also what you'd see if there were still a reader open.
Are you certain all readers were closed (finalized) when you ran lsof?

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message