lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shai Erera <ser...@gmail.com>
Subject Re: Extending Sort/FieldCache
Date Fri, 04 Sep 2009 16:19:51 GMT
Thanks Mike. I did not phrase well my understanding of Cache reload. I
didn't mean literally as part of the reopen, but *because* of the reopen.
Because FieldCache is tied to an IndexReader instance, after reopen it gets
refreshed. If I keep my own Cache, I'll need to code that logic, and I
prefer to use what's currently in Lucene. (Sloth? :) )

I'll look at the mentioned function queries. Thanks for the pointer !

Shai
On Fri, Sep 4, 2009 at 12:17 PM, Michael McCandless <
lucene@mikemccandless.com> wrote:

> On Fri, Sep 4, 2009 at 12:33 AM, Shai Erera<serera@gmail.com> wrote:
>
> > 1) Refactor the FieldCache API (and TopFieldCollector) such that one can
> > provide its own Cache of native values. I'd hate to rewrite the
> > FieldComparators logic just because the current API is not extendable.
> That
> > I agree should be quite straightforward, but I'll need to look into it
> > first.
>
> +1
>
> Or, so that the current consumers of FieldCache (that hardwire to
> FieldCache.DEFAULT) could simply accept any other FieldCache
> instance, as Hoss suggested.
>
> > 2) Contribute my payload-based sorting package. Currently it only reads
> from
> > disk during searches, and I'd like to enhance it to use in-memory cache
> as
> > well. It's a moderate-size package, so this one will need to wait until
> (1)
> > is done, and I get enough time to adapt it to 2.9 and work on the issue.
>
> +1, sounds cool!
>
> > I definitely don't think this should be a 2.9 sort of work, unless
> someone
> > else volunteers to do it. I can't pull (1) together that quickly.
>
> Me too... 2.9 train is moving out of the station now!
>
> > I thought that FieldCache is loaded when the IndexReader is reopened no?
> I
> > mean, it's a per IndexReader instance no? If not, then I'll need to
> rethink
> > the entire approach, because then (1) may not be that important anymore.
>
> FieldCache is only opened on demand, ie, the first time that a newly
> opened reader calls FieldCache.DEFAULT.getXXX does it get populated.
>
> So it's not during the reopen... it's during the first query that eg sorts
> by a field.
>
> It'd be great if your use case gives us enough clarity on LUCENE-831
> to make concrete progress.
>
> Note that function queries already "wrap" FieldCache w/ an alternate API
> for native value per doc (ValueSource, DocValues, FieldCacheSource),
> that do allow custom ValueSources to be provided.  It'd be great if
> as part of LUCENE-831 they could be more unified w/ FieldCache's
> API.
>
> Mike
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message