Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
MIME-Version: 1.0
In-Reply-To: <63e2e4460904110304n51de11f9m1f0daaa1e84db680@mail.gmail.com>
References: <70422ecc0904100738v23806a91gba4fca747ba3748f@mail.gmail.com>
	 <9ac0c6aa0904100748i65219c81k39274e15785ac082@mail.gmail.com>
	 <70422ecc0904101020k17cab6a8xe2365c1f2b5156b3@mail.gmail.com>
	 <9ac0c6aa0904101054o6cbfe83dhe0f2956151580f53@mail.gmail.com>
	 <70422ecc0904110006v35fdc65q77b939f849606b98@mail.gmail.com>
	 <85719C3168944A0DA8698E10398B86FE@VEGA>
	 <70422ecc0904110227u7870fb51wc20b9fc6501f35a6@mail.gmail.com>
	 <9ac0c6aa0904110248s4fbf83b6nafd28cc9b16b0c46@mail.gmail.com>
	 <63e2e4460904110304n51de11f9m1f0daaa1e84db680@mail.gmail.com>
Date: Sat, 11 Apr 2009 07:50:12 -0400
Message-ID: <9ac0c6aa0904110450h4d69905aw1f6d093698bb2528@mail.gmail.com>
Subject: Re: RangeFilter performance problem using MultiReader
From: Michael McCandless <lucene@mikemccandless.com>
To: java-user@lucene.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Hmm then I'm a bit baffled again.

Because, each of your "by month" indexes presumably has a unique
subset of terms for the "date_doc" field?  Meaning, a given "by month"
index will have all date_doc corresponding to that month, and a
different "by month" index would presumably have no overlap in the
terms for the date_doc field.

I'm baffled because I'd expect in such a case for the performance an
optimized index vs 72-sub-index MultiReader to be closer, on 2.9.
Hmm.

Mike

On Sat, Apr 11, 2009 at 6:04 AM, Roberto Franchini
<ro.franchini@gmail.com> wrote:
> On Sat, Apr 11, 2009 at 11:48 AM, Michael McCandless
> <lucene@mikemccandless.com> wrote:
>> On Sat, Apr 11, 2009 at 5:27 AM, Raf <r.ventaglio@gmail.com> wrote:
>>
> [cut]
>>
>> You have readers from 72 different directories, but is each directory
>> an optimized or unoptimized index?
>
> Hi,
> I'm Raffaella's collegue, and I'm the "indexer" while she is the "searcher" :)
>
> All indexes are optimized, we don't use compound files and we use the
> NIO FSDirectory, since we are on linux-64.
> We build indexes by month and then we aggregate them using the
> multireader. On every month we can have more than one index, e.g:
> 200901-->index1, index2
> 200902-->index3
> 200903-->index4,index5,index6
>
> Single indexes can be very little or very big, it depends on the source.
> I think we can try to merge on moth basis so we can obtain only one
> index for every month.
> 200901-->index12
> 200902-->index3
> 200903-->index456
>
> Doing so, each month's index will be quite the same in size. I tihink
> this could help to improve performance on 2.9.
>
> [cut]
>
> Thanks a lot to everybody.
> R.
>
> --
> Roberto Franchini
> http://www.celi.it
> http://www.blogmeter.it
> http://www.memesphere.it
> Tel +39-011-6600814
> jabber:ro.franchini@gmail.com skype:ro.franchini
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org