lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Elschot <paul.elsc...@xs4all.nl>
Subject Re: Using filters to speed up queries
Date Sun, 24 Oct 2010 09:27:52 GMT
Op zondag 24 oktober 2010 00:18:48 schreef Khash Sajadi:
> My index contains documents for different users. Each document has the user
> id as a field on it.
> 
> There are about 500 different users with 3 million documents.
> 
> Currently I'm calling Search with the query (parsed from user)
> and FieldCacheTermsFilter for the user id.
> 
> It works but the performance is not great.
> 
> Ideally, I would like to perform the search only on the documents that are
> relevant, this should make it much faster. However, it seems Search(Query,
> Filter) runs the query first and then applies the filter.
> 
> Is there a way to improve this? (i.e. run the query only on a subset of
> documents)
> 
> Thanks
> 

When running the query with the filter, the query is run at the same time
as the filter. Initially and after each matching document, the filter is assumed to
be cheaper to execute and its first or next matching document is determined.
Then the query and the filter are repeatedly advanced to each other's next matching
document until they are at the same document (ie. there is a match), similar to
a boolean query with two required clauses.
The java code doing this is in the private method IndexSearcher.searchWithFilter().

It could be that filling the field cache is the performance problem.
How is the performance when this search call with the FieldCacheTermsFilter
is repeated?

Also, for a single indexed term to be used as a filter (the user id in this case)
there may be no need for a cache, a QueryWrapperFilter around the TermQuery
might suffice.

Regards,
Paul Elschot

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message