lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Elschot <paul.elsc...@xs4all.nl>
Subject Re: [jira] Commented: (LUCENE-584) Decouple Filter from BitSet
Date Wed, 30 Aug 2006 21:24:06 GMT
Yonik,

issues.apache.org does not respond at the moment, so I'm answering here:

> It looks like no Filters currently return a matcher, so the current patch 
just lays the groundwork, right?

Right. Only the previous Filter-20060628.patch contains some commented FIXME 
code to actually introduce a BitsMatcher in each case where a BitSet is used.

>When some filters do start to return a matcher, it looks like support for the 
1.4 BooleanScorer needs
> to be removed, or a check done in IndexSearcher.search() to disable skipping 
on the scorer if it's in use.

Iirc the patch still supports the 1.4 BooleanScorer when a BitSet is returned 
by Filter. I'd have to have a look at the patched IndexSearcher to be sure 
though.
A BitSet is randomly addressable, so it can work to filter the 1.4 
BooleanScorer which can score documents out of order.  This case can be 
deprecated completely by also deprecating the possibility to use the 1.4 
boolean scorer, but that is not in the patch. The patch only deprecates the 
Filter.bits() method.


> I wonder what the performance impact is... for a dense search with a dense 
bitset
> filter, it looks like quite a bit of overhead is added (two calls in order 
to get the next 
> doc, use of nextSetBit() instead of get(), checking "exhausted" each time 
and 
> checking for -1 to set exhausted). I suppose one can always drop back to 
using
> a HitCollector for special cases though.

BitsMatcher could also work without the "exhausted" flag, but then an infinite 
loop
might occur when trying to continue after the first time next() or skipTo() 
returned false.
Continuing after false was returned in these cases is a bug, however an 
infinite loop
can be difficult to debug. I'd rather be on the safe side of that with the 
exhausted flag and wait for an actual profile to show the performance 
problem.

Regards,
Paul Elschot

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message