lucene-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Hostetter <>
Subject Re: Optimizing multi fields searches
Date Fri, 15 Oct 2010 21:26:59 GMT

: I have documents with 10 indexed fields and 10 million documents. 
: One of the index fields has ID-type value which is almost unique. when I
: search with that almost-unique field it comes up in 12ms, but if I add
: another criteria to the search it's very slow, like 5 secs. 
: I don't use QueryParser but I created my own queries with BooleanQuery of
: course. Is there anyway to optimize this? How can I influence Lucene to
: search index that yields less documents first and followed by the other
: indexes (the same concept used by some RDBMS that collects indexes
: statistics/histogram)?

BooleanQuery should already do this ... if you look into the code, the 
"skipTo" methods in the ConjunctionScorer API are where the logic takes 
place -- each BooleanClause is asked to find the "first" document that it 
matches and then the others are told to "skipTo" the first match they have 
which that doc or later -- so a query that matches only one doc will 
result in the other clauses skipping to that doc.

Since you haven't posted any code, it's hard to guess what might be going 
wrong for you -- but off the top of my head i can't help but wonder if you 
remembered to construct your BooleanClauses with Order.MUST (if you use 
Order.SHOULD then docs which match *either* clause will match ... and it's 
totally understandable that the query might be much slower)


View raw message