lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mark Harwood (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-584) Decouple Filter from BitSet
Date Thu, 26 Jul 2007 04:09:31 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12515494
] 

Mark Harwood commented on LUCENE-584:
-------------------------------------

Thanks for the reply, Paul.

I saw BitSetMatcher etc and appreciate the motivation behind the design for alternative implementations
. What concerns me with the Matcher API in general is that Matchers have non-threadsafe safe
state (i.e. the current position required to support next() )and as such aren't safely cachable
in the same way as BitSets. I see the searcher code uses the safer skipTo() rather than next()
 but there's still the "if(exhausted)" thread safety problem to worry about which is why I
raised points 1 and 4.

Additionally, combining Bitsets using Booolean logic is one method call whereas combining
heterogenous Matchers using Boolean logic requires iteration across them and therefore potentially
many method calls (point 3). I haven't benchmarked this but I imagine it to be significantly
slower?
I use BooleanFilter a lot for security where many large sets are cached and combined on the
fly - caching all the possible combinations as single bitsets would lead to too many possible
combinations. 

Cheers
Mark

> Decouple Filter from BitSet
> ---------------------------
>
>                 Key: LUCENE-584
>                 URL: https://issues.apache.org/jira/browse/LUCENE-584
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>    Affects Versions: 2.0.1
>            Reporter: Peter Schäfer
>            Priority: Minor
>         Attachments: bench-diff.txt, bench-diff.txt, Matcher-core20070725.patch, Matcher-default20070725.patch,
Matcher-ground20070725.patch, Some Matchers.zip
>
>
> {code}
> package org.apache.lucene.search;
> public abstract class Filter implements java.io.Serializable 
> {
>   public abstract AbstractBitSet bits(IndexReader reader) throws IOException;
> }
> public interface AbstractBitSet 
> {
>   public boolean get(int index);
> }
> {code}
> It would be useful if the method =Filter.bits()= returned an abstract interface, instead
of =java.util.BitSet=.
> Use case: there is a very large index, and, depending on the user's privileges, only
a small portion of the index is actually visible.
> Sparsely populated =java.util.BitSet=s are not efficient and waste lots of memory. It
would be desirable to have an alternative BitSet implementation with smaller memory footprint.
> Though it _is_ possibly to derive classes from =java.util.BitSet=, it was obviously not
designed for that purpose.
> That's why I propose to use an interface instead. The default implementation could still
delegate to =java.util.BitSet=.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message