lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <>
Subject Re: Extending Sort/FieldCache
Date Fri, 04 Sep 2009 09:17:29 GMT
On Fri, Sep 4, 2009 at 12:33 AM, Shai Erera<> wrote:

> 1) Refactor the FieldCache API (and TopFieldCollector) such that one can
> provide its own Cache of native values. I'd hate to rewrite the
> FieldComparators logic just because the current API is not extendable. That
> I agree should be quite straightforward, but I'll need to look into it
> first.


Or, so that the current consumers of FieldCache (that hardwire to
FieldCache.DEFAULT) could simply accept any other FieldCache
instance, as Hoss suggested.

> 2) Contribute my payload-based sorting package. Currently it only reads from
> disk during searches, and I'd like to enhance it to use in-memory cache as
> well. It's a moderate-size package, so this one will need to wait until (1)
> is done, and I get enough time to adapt it to 2.9 and work on the issue.

+1, sounds cool!

> I definitely don't think this should be a 2.9 sort of work, unless someone
> else volunteers to do it. I can't pull (1) together that quickly.

Me too... 2.9 train is moving out of the station now!

> I thought that FieldCache is loaded when the IndexReader is reopened no? I
> mean, it's a per IndexReader instance no? If not, then I'll need to rethink
> the entire approach, because then (1) may not be that important anymore.

FieldCache is only opened on demand, ie, the first time that a newly
opened reader calls FieldCache.DEFAULT.getXXX does it get populated.

So it's not during the reopen... it's during the first query that eg sorts
by a field.

It'd be great if your use case gives us enough clarity on LUCENE-831
to make concrete progress.

Note that function queries already "wrap" FieldCache w/ an alternate API
for native value per doc (ValueSource, DocValues, FieldCacheSource),
that do allow custom ValueSources to be provided.  It'd be great if
as part of LUCENE-831 they could be more unified w/ FieldCache's


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message