lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Istvan Soos <istvan.s...@gmail.com>
Subject Re: best practice on too many files vs IO overhead
Date Fri, 27 Nov 2009 10:48:59 GMT
On Fri, Nov 27, 2009 at 11:37 AM, Michael McCandless
<lucene@mikemccandless.com> wrote:
> Are you sure you're closing all readers that you're opening?

Absolutely. :) (okay, never say this, but I had bugz because of this
previously so I'm pretty sure that one is ok).

> It's surprising with normal usage of Lucene that you'd run out of
> descriptors, with its default mergeFactor (have you increased the
> mergeFactor)?

Default merge factor. (on Mac, the default maxfiles is 256, however
I've run out of descriptors event at 10240, if I hadn't called
optimize).

> You can also enable compound file, which uses far fewer file
> descriptors, at some cost to indexing performance.

I thought this is the default but I'll check...

> Also, a partial optimize (ie optimize(N)) does less IO but still
> substantially reduces segment count of the index.

I wasn't aware of this, thanks, I'll try it!

Regards,
   Istvan

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message