lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler (JIRA)" <>
Subject [jira] Commented: (LUCENE-1461) Cached filter for a single term field
Date Fri, 26 Jun 2009 15:39:07 GMT


Uwe Schindler commented on LUCENE-1461:

>From your performance runs, looking at the average times, forcing this
filter to take deletions into account made it ~2X slower. That's
quite costly.

This is not only because of the deleted docs. Its also because the while loops are no longer
hard coded with bounds, so there is an additional method call to check if something is a hit.
The linear scan makes this also very costly. But without it, the code is unmaintainable (with
all problems like copy/paste errors) and so on. It is almost 6 times the same code :(

I would imagine that for most usage of this filter, taking deletes
into account is not necessary, because it's being used as a filter
with a query whose scorer won't return deleted docs. Then we've taken
this perf hit for nothing...

I could put in a switch, that uses another iterator, if 0 is not inside the range (because
then all deleted docs would never be hits) or the IR has no deletions. But I think this optimization
is something for later.

In my opinion, in most cases TrieRange is better (and with Payloads, too). So I keeps this
filter how it is for the beginning.

> Cached filter for a single term field
> -------------------------------------
>                 Key: LUCENE-1461
>                 URL:
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Tim Sturge
>            Assignee: Uwe Schindler
>             Fix For: 2.9
>         Attachments:, FieldCacheRangeFilter.patch, LUCENE-1461.patch,
LUCENE-1461.patch, LUCENE-1461.patch, LUCENE-1461.patch, LUCENE-1461.patch, LUCENE-1461.patch,
LUCENE-1461a.patch, LUCENE-1461b.patch, LUCENE-1461c.patch,,,,, TestFieldCacheRangeFilter.patch
> These classes implement inexpensive range filtering over a field containing a single
term. They do this by building an integer array of term numbers (storing the term->number
mapping in a TreeMap) and then implementing a fast integer comparison based DocSetIdIterator.
> This code is currently being used to do age range filtering, but could also be used to
do other date filtering or in any application where there need to be multiple filters based
on the same single term field. I have an untested implementation of single term filtering
and have considered but not yet implemented term set filtering (useful for location based
searches) as well. 
> The code here is fairly rough; it works but lacks javadocs and toString() and hashCode()
methods etc. I'm posting it here to discover if there is other interest in this feature; I
don't mind fixing it up but would hate to go to the effort if it's not going to make it into

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message