lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Elschot <>
Subject Re: [jira] Commented: (LUCENE-584) Decouple Filter from BitSet
Date Wed, 30 Aug 2006 21:24:06 GMT
Yonik, does not respond at the moment, so I'm answering here:

> It looks like no Filters currently return a matcher, so the current patch 
just lays the groundwork, right?

Right. Only the previous Filter-20060628.patch contains some commented FIXME 
code to actually introduce a BitsMatcher in each case where a BitSet is used.

>When some filters do start to return a matcher, it looks like support for the 
1.4 BooleanScorer needs
> to be removed, or a check done in to disable skipping 
on the scorer if it's in use.

Iirc the patch still supports the 1.4 BooleanScorer when a BitSet is returned 
by Filter. I'd have to have a look at the patched IndexSearcher to be sure 
A BitSet is randomly addressable, so it can work to filter the 1.4 
BooleanScorer which can score documents out of order.  This case can be 
deprecated completely by also deprecating the possibility to use the 1.4 
boolean scorer, but that is not in the patch. The patch only deprecates the 
Filter.bits() method.

> I wonder what the performance impact is... for a dense search with a dense 
> filter, it looks like quite a bit of overhead is added (two calls in order 
to get the next 
> doc, use of nextSetBit() instead of get(), checking "exhausted" each time 
> checking for -1 to set exhausted). I suppose one can always drop back to 
> a HitCollector for special cases though.

BitsMatcher could also work without the "exhausted" flag, but then an infinite 
might occur when trying to continue after the first time next() or skipTo() 
returned false.
Continuing after false was returned in these cases is a bug, however an 
infinite loop
can be difficult to debug. I'd rather be on the safe side of that with the 
exhausted flag and wait for an actual profile to show the performance 

Paul Elschot

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message