lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: Deleted File Handles - Index Writer
Date Fri, 19 Nov 2010 10:46:30 GMT
On Fri, Nov 19, 2010 at 5:06 AM, Thomas Rewig <trewig@mufin.com> wrote:
>  Hi.
>>>
>>> If I set
>>> writer.setUseCompoundFile(false);
>>> than I see a short time the 35 handles and than it drops to zero.
>>
>> Hmm what's happening during that short time?  Do you have a reader open?
>>
> Yes after a commit has been initiated, a short time a reader is still open
> and that's ok. This behavior is therefore expected if I have understood all
> correctly.

OK excellent.

>>>
>>> If I do not use the compound file format as my new default until the new
>>> version with the fix is available, what would be the disadvantages?
>>
>> The big issue with CFS turned off is you risk running out of file
>> descriptors if you have too many segments in your index.  The exact
>> limit is very OS dependent though, so if you're on an OS that's
>> generous, and mergeFactor stays low, then you should be fine.
>>
> Ok I think the limit of file descriptors would be a problem ... there are 5
> Liveindices.
>
> At the moment I do not change the MergeFactor. So I think the
> DEFAULT_MERGE_FACTOR with 10 is used. The only thing I changed is:
>
> writer = new IndexWriter(directory, analyzer,
> create,IndexWriter.MaxFieldLength.UNLIMITED);
> writer.setRAMBufferSizeMB(48);
>
> I think it is to risky to turn of the CFS, because I have no experience with
> MergeFactor and the number of descriptors 5 Liveindices will use.
> At the moment it looks as if I does not bump with the current configuration
> into any limits, as long as I change the whole Index every week. I leave the
> settings with CFS and wait for the fix.

Likely you'd be fine with no CFS, but I understand the risk
aversion... it's a lesser of evils situation.

The open but deleted file limit is bounded by the number of segments
in the index.

> Is there a way to patch the 3.0.2
> Release version or have I wait for a future release?

I haven't yet committed the fix... when I do (soon!), I'll backport to
3.0.x and 2.9.x, and then you can just get a source checkout of that
branch and build the JAR from it, eg:

  svn checkout https://svn.apache.org/repos/asf/lucene/java/branches/lucene_3_0
  cd lucene_3_0
  ant jar
  cp build/lucene-core-3.0.3-dev.jar /path/to/your/lucene/jar

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message