lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wettin <karl.wet...@gmail.com>
Subject Re: Boolean Query search performance
Date Thu, 06 Mar 2008 01:01:29 GMT
Beard, Brian skrev:
> I'm using lucene 2.2.0.
> 
> I'm in the process of re-writing some queries to build BooleanQueries
> instead of using query parser.
> 
> Bypassing query parser provides almost an order of magnitude improvement
> for very large queries, but then the search performance takes 20-30%
> longer.

Have you benchmarked the time spent creating the query and the time 
spent searching using the query seperatly? QueryParser can be sort of 
expensive. It also seems to me that you are creating a very large query 
given you add one criteria per entity id.

     karl



> I'm adding boost values as a way to sort (long story, no
> workaround I can see for now).
> 
> If I do a query.toString(), both queries give different results, which
> is probably a clue (additional paren's with the BooleanQuery)
> 
> Query.toString the old way using queryParser:
>     +(id:1^2.0 id:2 ... ) +type:CORE
> 
> Query.toString the new way using BooleanQuery:
>     +((id:1^2.0) (id:2) ... ) +type:CORE
> 
> Does anyone have any ideas as to why the performance difference in
> searching. I would think I could get the search time to be the same,
> since after the query is formed everything remains the same. The same
> number of hits are going through the hitCollector that gets called after
> this, so all variables seem constant. QueryParser must be doing some
> optimization I'm not taking advantage of. Code snippet follows.....
> 
> The previous way using queryParser was:
> 
>   String queryStr = (id:1^2 OR id:2 .... id:n) AND type:CORE
>   Query query = parser.parse(queryStr);
> 
> 
> The new way using BooleanQuery is...
> 
>   BooleanQuery totalBooleanQuery = new BooleanQuery();
>   BooleanClause docTypeClause = 
>     new BooleanClause( new TermQuery( 
>     new Term("type", "CORE")), BooleanClause.Occur.MUST);
> 
>   BooleanQuery idBooleanQuery = new BooleanQuery();
>   for (String uid : uniqueIdMap.keySet()) {
> 	TermQuery termQuery = 
>          new TermQuery(new Term("id", uid));
> 	termQuery.setBoost(loopBoostValue);
> 	idBooleanQuery.add(
>          new BooleanClause(termQuery, BooleanClause.Occur.SHOULD));
>   } 
> 
>   totalBooleanQuery.add(docTypeClause);
>   totalBooleanQuery.add(idBooleanQuery, Occur.MUST); 
> 
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message