Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (nike.apache.org: domain of istvan.soos@gmail.com
 designates 209.85.220.224 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=tZcwR9hupncZ817MwBhkB0ZrFDnKLbDJJ6ZmWR2Qmo/yceACJX02DTKczO2yGf9npc
         UltikrTaXhbKUv08pBPchkDlZ5Ndx4qtQI91hlHzf+jrWz+x5ptshpTp2uXp05YELQn5
         +ZUTIxSQkTHdnjJm1LBOF8iNRjr2NY+h6FJTw=
MIME-Version: 1.0
In-Reply-To: <9ac0c6aa0911270237w4929254dp702cdda50548f603@mail.gmail.com>
References: <7ac523660911270123p6f16a385rfafd5b80307ccd90@mail.gmail.com>
	 <9ac0c6aa0911270237w4929254dp702cdda50548f603@mail.gmail.com>
Date: Fri, 27 Nov 2009 11:48:59 +0100
Message-ID: <7ac523660911270248n1b6af210u944d2d238e945bda@mail.gmail.com>
Subject: Re: best practice on too many files vs IO overhead
From: Istvan Soos <istvan.soos@gmail.com>
To: java-user@lucene.apache.org
Content-Type: text/plain; charset=ISO-8859-1

On Fri, Nov 27, 2009 at 11:37 AM, Michael McCandless
<lucene@mikemccandless.com> wrote:
> Are you sure you're closing all readers that you're opening?

Absolutely. :) (okay, never say this, but I had bugz because of this
previously so I'm pretty sure that one is ok).

> It's surprising with normal usage of Lucene that you'd run out of
> descriptors, with its default mergeFactor (have you increased the
> mergeFactor)?

Default merge factor. (on Mac, the default maxfiles is 256, however
I've run out of descriptors event at 10240, if I hadn't called
optimize).

> You can also enable compound file, which uses far fewer file
> descriptors, at some cost to indexing performance.

I thought this is the default but I'll check...

> Also, a partial optimize (ie optimize(N)) does less IO but still
> substantially reduces segment count of the index.

I wasn't aware of this, thanks, I'll try it!

Regards,
   Istvan

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org