lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: RangeFilter performance problem using MultiReader
Date Sat, 11 Apr 2009 09:48:59 GMT
On Sat, Apr 11, 2009 at 5:27 AM, Raf <r.ventaglio@gmail.com> wrote:

> I have repeated my tests using a searcher and now the performance on 2.9 are
> very better than those on 2.4.1, especially when the filter extracts a lot
> of docs.

OK, phew!

> However the same search on the consolidated index is even faster

This is very interesting... I think it must be because your field's
terms have some redundancy (299,622 unique terms vs ~3M docs), so
consolidation does improve performance substantially.

You have readers from 72 different directories, but is each directory
an optimized or unoptimized index?

> now to verify if it is better for us to spend more time in creating indexes
> (i.e. to add the overhead of consolidation) and to have very fast range
> filter searches or to keep our small indexes and to have less fast range
> filter searches.

On 2.9, I would first try LongTrieRangeFilter... it's simple to use
and may give you sizable performance gains on the MultiReader case
(and, on the consolidated index case).

> In any case, I must wait until lucene 2.9 will be officially released,
> before put it on the production environment, so I think I will have to
> consolidate indexes for now.

OK... there're alot of good changes coming up in 2.9 so hopefully we
can get a release out soon.

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message