lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: Extending Sort/FieldCache
Date Sat, 05 Sep 2009 00:52:32 GMT
No, not sloth. Making use of the fine work that others have done in
order to help get your product out the door faster/cheaper <G>....

As in "There's no virtue in re-inventing the wheel. No matter how
productive it feels" <GGGG>.

Best
Erick

On Fri, Sep 4, 2009 at 12:19 PM, Shai Erera <serera@gmail.com> wrote:

> Thanks Mike. I did not phrase well my understanding of Cache reload. I
> didn't mean literally as part of the reopen, but *because* of the reopen.
> Because FieldCache is tied to an IndexReader instance, after reopen it gets
> refreshed. If I keep my own Cache, I'll need to code that logic, and I
> prefer to use what's currently in Lucene. (Sloth? :) )
>
> I'll look at the mentioned function queries. Thanks for the pointer !
>
> Shai
> On Fri, Sep 4, 2009 at 12:17 PM, Michael McCandless <
> lucene@mikemccandless.com> wrote:
>
> > On Fri, Sep 4, 2009 at 12:33 AM, Shai Erera<serera@gmail.com> wrote:
> >
> > > 1) Refactor the FieldCache API (and TopFieldCollector) such that one
> can
> > > provide its own Cache of native values. I'd hate to rewrite the
> > > FieldComparators logic just because the current API is not extendable.
> > That
> > > I agree should be quite straightforward, but I'll need to look into it
> > > first.
> >
> > +1
> >
> > Or, so that the current consumers of FieldCache (that hardwire to
> > FieldCache.DEFAULT) could simply accept any other FieldCache
> > instance, as Hoss suggested.
> >
> > > 2) Contribute my payload-based sorting package. Currently it only reads
> > from
> > > disk during searches, and I'd like to enhance it to use in-memory cache
> > as
> > > well. It's a moderate-size package, so this one will need to wait until
> > (1)
> > > is done, and I get enough time to adapt it to 2.9 and work on the
> issue.
> >
> > +1, sounds cool!
> >
> > > I definitely don't think this should be a 2.9 sort of work, unless
> > someone
> > > else volunteers to do it. I can't pull (1) together that quickly.
> >
> > Me too... 2.9 train is moving out of the station now!
> >
> > > I thought that FieldCache is loaded when the IndexReader is reopened
> no?
> > I
> > > mean, it's a per IndexReader instance no? If not, then I'll need to
> > rethink
> > > the entire approach, because then (1) may not be that important
> anymore.
> >
> > FieldCache is only opened on demand, ie, the first time that a newly
> > opened reader calls FieldCache.DEFAULT.getXXX does it get populated.
> >
> > So it's not during the reopen... it's during the first query that eg
> sorts
> > by a field.
> >
> > It'd be great if your use case gives us enough clarity on LUCENE-831
> > to make concrete progress.
> >
> > Note that function queries already "wrap" FieldCache w/ an alternate API
> > for native value per doc (ValueSource, DocValues, FieldCacheSource),
> > that do allow custom ValueSources to be provided.  It'd be great if
> > as part of LUCENE-831 they could be more unified w/ FieldCache's
> > API.
> >
> > Mike
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message