lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <>
Subject Re: Deleted File Handles - Index Writer
Date Fri, 19 Nov 2010 10:46:30 GMT
On Fri, Nov 19, 2010 at 5:06 AM, Thomas Rewig <> wrote:
>  Hi.
>>> If I set
>>> writer.setUseCompoundFile(false);
>>> than I see a short time the 35 handles and than it drops to zero.
>> Hmm what's happening during that short time?  Do you have a reader open?
> Yes after a commit has been initiated, a short time a reader is still open
> and that's ok. This behavior is therefore expected if I have understood all
> correctly.

OK excellent.

>>> If I do not use the compound file format as my new default until the new
>>> version with the fix is available, what would be the disadvantages?
>> The big issue with CFS turned off is you risk running out of file
>> descriptors if you have too many segments in your index.  The exact
>> limit is very OS dependent though, so if you're on an OS that's
>> generous, and mergeFactor stays low, then you should be fine.
> Ok I think the limit of file descriptors would be a problem ... there are 5
> Liveindices.
> At the moment I do not change the MergeFactor. So I think the
> DEFAULT_MERGE_FACTOR with 10 is used. The only thing I changed is:
> writer = new IndexWriter(directory, analyzer,
> create,IndexWriter.MaxFieldLength.UNLIMITED);
> writer.setRAMBufferSizeMB(48);
> I think it is to risky to turn of the CFS, because I have no experience with
> MergeFactor and the number of descriptors 5 Liveindices will use.
> At the moment it looks as if I does not bump with the current configuration
> into any limits, as long as I change the whole Index every week. I leave the
> settings with CFS and wait for the fix.

Likely you'd be fine with no CFS, but I understand the risk
aversion... it's a lesser of evils situation.

The open but deleted file limit is bounded by the number of segments
in the index.

> Is there a way to patch the 3.0.2
> Release version or have I wait for a future release?

I haven't yet committed the fix... when I do (soon!), I'll backport to
3.0.x and 2.9.x, and then you can just get a source checkout of that
branch and build the JAR from it, eg:

  svn checkout
  cd lucene_3_0
  ant jar
  cp build/lucene-core-3.0.3-dev.jar /path/to/your/lucene/jar


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message