lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Trejkaz <trej...@trypticon.org>
Subject Re: Does anyone have tips on managing cached filters?
Date Thu, 29 Nov 2012 05:57:03 GMT
On Wed, Nov 28, 2012 at 6:28 PM, Robert Muir <rcmuir@gmail.com> wrote:
> My point is really that lucene (especially clear in 4.0) assumes
> indexreaders are immutable points in time. I don't think it makes sense for
> us to provide any e.g. filtercaching or similar otherwise, because this is
> a key simplification to the design. If you depart from this, by scoring or
> filtering from mutable stuff outside the inverted index, things are likely
> going to get complicated.

Whereas it would be lovely to live in a land of rainbows and unicorns
where all the data you ever want to use is in the text index and all
filters can be written as a query, that simply isn't the case for us
and I very much doubt we're not the only ones in this situation.

Sure, things are complicated. Anything except the most trivial forum
search application is complicated.

Well, the situation as it stands now is that when a filter is
invalidated, it happens across all stores which are currently open.
That means that results are at least correct, but after invalidating a
filter, a little more work than necessary is required to populate the
cache again. For certain filters (like word lists) this is necessary
anyway, since adding a word might invalidate any store. For others
like tags, I was hoping there would be some way to selectively
invalidate only certain readers. But it seems like that isn't the
case, so I will probably have to add a third level of caching to cache
these sorts of filter per-store instead of globally.

TX

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message