lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Rewig <tre...@mufin.com>
Subject Re: Deleted File Handles - Index Writer
Date Thu, 18 Nov 2010 15:10:46 GMT
  Hi Michael,

Thanks for your answer and sorry for my late reply.
> Are you using compound file format (the default)?
>
Yes I am using the compound file format as default.
> If you turn that off (just for this test) do you still see that
> IndexWriter is holding open the files (35 in your example) after you
> close all readers?

If I set
writer.setUseCompoundFile(false);
than I see a short time the 35 handles and than it drops to zero.

If I do not use the compound file format as my new default until the new 
version with the fix is available, what would be the disadvantages?

Thanks for your help.
Thomas

> I've found a case, only with compound file, where IndexWriter holds
> open a SegmentReader on the pre-compound-file files... I'm working on
> a test case&  fix.
>
> Mike
>
> On Fri, Nov 12, 2010 at 5:49 AM, Thomas Rewig<trewig@mufin.com>  wrote:
>>   Hello,
>>
>> I use the searcherManager for LiveIndexing. With  watch -n 60 "lsof | grep
>> indexname | grep deleted | wc -l" I see the number of deleted file handles.
>> The number of handles fluctuates during the indexing.  0 ->  35 ->  53 ->
 135
>> ->  40 ->  85 ... Uwe said that this is expected because segments are still
>> referenced by the open IndexReaders, but files were already deleted by
>> IndexWriter.
>>
>> But something puzzles me:
>>
>> Let's say we have a deleted file handle number of 85. If I switch the
>> NearRealTime Searcher to the readOnlySearcher the file handle number is
>> still 85. If i close afterwards the no more used indexwriter, the number
>> drops to 35.  I think that means, that the indexwriter referenced deleted
>> files that are not referenced by the readers.  It seems, that with every
>> commit the number of deleted file handles (which maybe are "caused" by the
>> indexwriter) grows.
>> Is that possible and if yes why does the indexwriter do it? Is there a max
>> Value of deleted handles an IndexWriter could own, because I don't want to
>> chrash the system because of too much open filehandles?
>>
>> Thanks in advance.
>> Thomas
>> --
>>
>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message