lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Elschot <paul.elsc...@xs4all.nl>
Subject Re: chained restrictive queries
Date Mon, 14 Feb 2005 19:59:18 GMT
On Monday 14 February 2005 15:14, oquinton@viamichelin.com wrote:
> Hi,
> 
> I'm currently working on application using Lucene 1.3 , and have to improve
> the current indexation/search methods with the 1.4.3 version.
> 
> 
> I was thinking to use the FilteredQuery object to refine my chained queries
> but, after some tests, performances are worst :(.
>
> The chained queries were like :
> - a first boolean query to retrieve a set of doc id matching some criterias

A FilteredQuery works best when the filter from the criterias can be reused,
eg. by keeping it in a cache, possibly with CachingWrapperFilter.

> - a second query applying a fuzzy criteria to refine it more deeply.
> 
> My index contains like 7 millions of document at all , and first query
> should retrieve, at maximum, like 50 000 documents.
>
> I'm currently working with crossed indexes while doing searches , but i
> want to remove the extra indexes and do all things with only one.
> 
> So, is it possible to use the FilteredQuery object or another one to chain
> queries from the most restrictive to the most open one ?

It is possible, but whether it helps performance depends on your
circumstances.

The 1.4.3 filter implementation executes the most open query almost
completely.
It only applies the filter after the score computations for the
query being filtered, just before deciding whether to keep the docment
in the query results.
This is done in IndexSearcher.search(). 
A profiler might tell you whether that is a bottleneck for your queries.
If it is, there is some code in development that might help
.
In case it turns out that the memory occupied by the BitSet of the filter
is a bottleneck, please check the (very) recent archives of lucene-dev
on BitSet implementation.

Regards,
Paul Elschot


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Mime
View raw message