lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: RangeFilter performance problem using MultiReader
Date Sat, 11 Apr 2009 11:50:12 GMT
Hmm then I'm a bit baffled again.

Because, each of your "by month" indexes presumably has a unique
subset of terms for the "date_doc" field?  Meaning, a given "by month"
index will have all date_doc corresponding to that month, and a
different "by month" index would presumably have no overlap in the
terms for the date_doc field.

I'm baffled because I'd expect in such a case for the performance an
optimized index vs 72-sub-index MultiReader to be closer, on 2.9.
Hmm.

Mike

On Sat, Apr 11, 2009 at 6:04 AM, Roberto Franchini
<ro.franchini@gmail.com> wrote:
> On Sat, Apr 11, 2009 at 11:48 AM, Michael McCandless
> <lucene@mikemccandless.com> wrote:
>> On Sat, Apr 11, 2009 at 5:27 AM, Raf <r.ventaglio@gmail.com> wrote:
>>
> [cut]
>>
>> You have readers from 72 different directories, but is each directory
>> an optimized or unoptimized index?
>
> Hi,
> I'm Raffaella's collegue, and I'm the "indexer" while she is the "searcher" :)
>
> All indexes are optimized, we don't use compound files and we use the
> NIO FSDirectory, since we are on linux-64.
> We build indexes by month and then we aggregate them using the
> multireader. On every month we can have more than one index, e.g:
> 200901-->index1, index2
> 200902-->index3
> 200903-->index4,index5,index6
>
> Single indexes can be very little or very big, it depends on the source.
> I think we can try to merge on moth basis so we can obtain only one
> index for every month.
> 200901-->index12
> 200902-->index3
> 200903-->index456
>
> Doing so, each month's index will be quite the same in size. I tihink
> this could help to improve performance on 2.9.
>
> [cut]
>
> Thanks a lot to everybody.
> R.
>
> --
> Roberto Franchini
> http://www.celi.it
> http://www.blogmeter.it
> http://www.memesphere.it
> Tel +39-011-6600814
> jabber:ro.franchini@gmail.com skype:ro.franchini
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message