lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <>
Subject [jira] Commented: (LUCENE-831) Complete overhaul of FieldCache API/Implementation
Date Sun, 12 Apr 2009 19:25:14 GMT


Michael McCandless commented on LUCENE-831:

This issue is fast moving!  Here're some thoughts on the last patch:

  * Need to pass in ValueSource impl to, defaulting
    to Cached(Uninverted)

  * Maybe ValueSource should provide default "throws UOE" exceptions
    methods (instead of abstract) since it's rather cumbersome for a
    subclass that only intends to provide eg getInts().

  * Should CachingValueSource really include IndexReader in its key?
    I think it shouldn't?  The cache is already "scoped" to a reader
    because each SegmentReader will have its own instance.  Also, one
    might cache ValueSources that don't "have" a reader (eg pull from
    a DB or custom file format or something).  Cloning an SR for now
    should just copy over the same cache.

  * I think we should back deprecated FieldCache with ValueSource for
    the reader; one simple way to not ignore the Parser passed in is
    to anonymously subclass Uninverter and invoke the passed in
    ByteParser from its "newTerm" method?

  * Why do we need to define StringInverter, IntInverter, etc in
    UninversionValueSource?  Couldn't someone instead subclass
    UninversionValueSource, and override whichever getXXX's they want
    using their own anonymous uninverter subclass?

  * Should we deprecate function.FieldCacheSource, and make a new
    function.ReaderValueSource (ie pulls its values from the
    IndexReader's ValueSource)?

> Complete overhaul of FieldCache API/Implementation
> --------------------------------------------------
>                 Key: LUCENE-831
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>            Reporter: Hoss Man
>            Assignee: Mark Miller
>             Fix For: 3.0
>         Attachments:, fieldcache-overhaul.032208.diff, fieldcache-overhaul.diff,
fieldcache-overhaul.diff, LUCENE-831.03.28.2008.diff, LUCENE-831.03.30.2008.diff, LUCENE-831.03.31.2008.diff,
LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch,
LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch
> Motivation:
> 1) Complete overhaul the API/implementation of "FieldCache" type things...
>     a) eliminate global static map keyed on IndexReader (thus
>         eliminating synch block between completley independent IndexReaders)
>     b) allow more customization of cache management (ie: use 
>         expiration/replacement strategies, disk backed caches, etc)
>     c) allow people to define custom cache data logic (ie: custom
>         parsers, complex datatypes, etc... anything tied to a reader)
>     d) allow people to inspect what's in a cache (list of CacheKeys) for
>         an IndexReader so a new IndexReader can be likewise warmed. 
>     e) Lend support for smarter cache management if/when
>         IndexReader.reopen is added (merging of cached data from subReaders).
> 2) Provide backwards compatibility to support existing FieldCache API with
>     the new implementation, so there is no redundent caching as client code
>     migrades to new API.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message