lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mark Harwood (JIRA)" <>
Subject [jira] Commented: (LUCENE-584) Decouple Filter from BitSet
Date Wed, 08 Aug 2007 22:57:00 GMT


Mark Harwood commented on LUCENE-584:

Hi Paul,
Many thanks for your responses.
Sorry for the delay in communications - just got back from 2 weeks holiday and slowly picking
my way through this patch. 

You said: "there is never a threadsafety problem. (See BitSetMatcher.getMatcher() which uses
a local class for the resulting Matcher.)"

Did you mean BitSetFilter.getMatcher()? BitSetMatcher has no getMatcher method.

If so, doesn't my original thread safety issue still stand? - CachingWrapperFilter is caching
Matchers (not Filters which are factories for matchers). 

The existing approach of adding a <CachedFilter> tag around my XML-based query templates
offers a major speed up in my applications and I don't see this supported in this patch currently
which gives me some concern. This existing caching technique is based on the use of CachingWrapperFilter.

The proposed framework seems to be missing a means of caching reusable, threadsafe  Matchers
in a type-independent fashion. One solution (which I think you may be suggesting with the
"getMatcher" comment) is to cache Filter objects and use Filter.getMatcher(reader) as a factory
method for thread-specific, single-use Matchers but this would suggest that any caching then
becomes an implied responsibility/overhead of each Filter implementation. Not too great. CachingWrapperFilter
is an example of a better design where the caching policy has been implemented in a single
class and it can be used to decorate any Filter implementation (RangeFilter etc) with the
required caching behaviour. Unfortunately with this proposed patch there is no way that any
such single caching policy can work with any Filter because Matcher is not reusable/cachable.
Time to remove any  thread-specific state from Matcher?


> Decouple Filter from BitSet
> ---------------------------
>                 Key: LUCENE-584
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>    Affects Versions: 2.0.1
>            Reporter: Peter Schäfer
>            Priority: Minor
>         Attachments: bench-diff.txt, bench-diff.txt, Matcher1-ground-20070730.patch,
Matcher2-default-20070730.patch, Matcher3-core-20070730.patch, Matcher4-contrib-misc-20070730.patch,
Matcher5-contrib-queries-20070730.patch, Matcher6-contrib-xml-20070730.patch, Some
> {code}
> package;
> public abstract class Filter implements 
> {
>   public abstract AbstractBitSet bits(IndexReader reader) throws IOException;
> }
> public interface AbstractBitSet 
> {
>   public boolean get(int index);
> }
> {code}
> It would be useful if the method =Filter.bits()= returned an abstract interface, instead
of =java.util.BitSet=.
> Use case: there is a very large index, and, depending on the user's privileges, only
a small portion of the index is actually visible.
> Sparsely populated =java.util.BitSet=s are not efficient and waste lots of memory. It
would be desirable to have an alternative BitSet implementation with smaller memory footprint.
> Though it _is_ possibly to derive classes from =java.util.BitSet=, it was obviously not
designed for that purpose.
> That's why I propose to use an interface instead. The default implementation could still
delegate to =java.util.BitSet=.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message