lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shai Erera <>
Subject Re: Extending Sort/FieldCache
Date Fri, 04 Sep 2009 04:33:10 GMT

I plan to look into two things, and then probably create two separate

1) Refactor the FieldCache API (and TopFieldCollector) such that one can
provide its own Cache of native values. I'd hate to rewrite the
FieldComparators logic just because the current API is not extendable. That
I agree should be quite straightforward, but I'll need to look into it

2) Contribute my payload-based sorting package. Currently it only reads from
disk during searches, and I'd like to enhance it to use in-memory cache as
well. It's a moderate-size package, so this one will need to wait until (1)
is done, and I get enough time to adapt it to 2.9 and work on the issue.

I definitely don't think this should be a 2.9 sort of work, unless someone
else volunteers to do it. I can't pull (1) together that quickly.

I thought that FieldCache is loaded when the IndexReader is reopened no? I
mean, it's a per IndexReader instance no? If not, then I'll need to rethink
the entire approach, because then (1) may not be that important anymore.


On Fri, Sep 4, 2009 at 1:15 AM, Chris Hostetter <>wrote:

> : I wanted to avoid two things:
> : * Writing the logic that invokes cache-refresh upon IndexReader reload.
> Uh... i don't think there is any code that FieldCache refreshing on
> reload (yet), so you wouldn't be missing out on anything.  (as long as
> your custom cache works at the SegmentReader level, you'd have the exact
> same benefits as using FieldCache)
> : * Write my own TopFieldCollector which uses this cache.
> well, yeah ... there you're screwed.
> you know ... just because FieldCache.DEFAULT is a static final singleton,
> doesn't mean we need to have FieldCache.DEFAULT hardcoded in so many
> places ... a lot of the high level classes that refer to
> FieldCache.DEFAULT could probably be refacotred to have a
> setFieldCache(FieldCache) method that defaults to FieldCache.DEFAULT, and
> then they could pass it down to the lower level classes ... that way you
> *could* write a "MyFieldCache extends FieldCache", and then have a
> SortComparator (or FieldComparator, or whatever it's called now) that cast
> the FieldCache it gets to your MyFieldCache and calls the custom methods
> ... withoutneeding to rewrite all of hte Collector code.
> It sounds a little invasive to make it into 2.9 ... but i haven't looked
> ito what it would involve ... in theory it could be really straight
> forward and backwards compatible.
> -Hoss
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message