Mailing-List: contact java-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-dev@lucene.apache.org
Message-ID: <1976905870.1239835635157.JavaMail.jira@brutus>
Date: Wed, 15 Apr 2009 15:47:15 -0700 (PDT)
From: "Mark Miller (JIRA)" <jira@apache.org>
To: java-dev@lucene.apache.org
Subject: [jira] Updated: (LUCENE-831) Complete overhaul of FieldCache
 API/Implementation
In-Reply-To: <19567919.1173857169285.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/LUCENE-831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mark Miller updated LUCENE-831:
-------------------------------

    Attachment: LUCENE-831.patch

I've added your SortField constructor Uwe.

I've also switched things to what I proposed.

You can access a ValueSource from SegmentReaders with getValueSource. You get a USO Exception on non SegmentReaders with the hint to use getSequentialSubReaders.
The built in FieldCache usage uses this new API instead, allowing caching per segment without global caching keyed by IndexReader. Its hardwired at the moment, but you would be able to set it on IndexReader open.

There is a back compat CacheByReaderUninverterValueSource that caches in a WeakHashMap by IndexReader and Uninverter (Uninverter is actually keyed by the Parser it wraps). This
works as a Singleton. Deprecated code, like the FieldCache uses this. You wouldn't want to straddle both API's, because for identical types / uninversion you would duplicate data (eg if you asked both APIs for the same data type with
the same parser/uninverter and field, you would get twice the data in RAM). This method still works with MultiReaders, and so nicely solves back compat with regards to only allowing access to ValueSource from the SegmentReader.

You can also set a ValueSource on a SortField. This would override the ValueSource used at sort time to one provided. There is a convenience class called CacheByReaderValueSource that caches
by Reader, field, type, key.

I think that solves all three needs (and my caching concerns) in a pretty nice way. Patch is not done, but all tests now pass except for TrieRange (have not run back compat tests yet due to Trie failure - havnt looked into yet either).

> Complete overhaul of FieldCache API/Implementation
> --------------------------------------------------
>
>                 Key: LUCENE-831
>                 URL: https://issues.apache.org/jira/browse/LUCENE-831
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>            Reporter: Hoss Man
>            Assignee: Mark Miller
>             Fix For: 3.0
>
>         Attachments: ExtendedDocument.java, fieldcache-overhaul.032208.diff, fieldcache-overhaul.diff, fieldcache-overhaul.diff, LUCENE-831-trieimpl.patch, LUCENE-831.03.28.2008.diff, LUCENE-831.03.30.2008.diff, LUCENE-831.03.31.2008.diff, LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch
>
>
> Motivation:
> 1) Complete overhaul the API/implementation of "FieldCache" type things...
>     a) eliminate global static map keyed on IndexReader (thus
>         eliminating synch block between completley independent IndexReaders)
>     b) allow more customization of cache management (ie: use 
>         expiration/replacement strategies, disk backed caches, etc)
>     c) allow people to define custom cache data logic (ie: custom
>         parsers, complex datatypes, etc... anything tied to a reader)
>     d) allow people to inspect what's in a cache (list of CacheKeys) for
>         an IndexReader so a new IndexReader can be likewise warmed. 
>     e) Lend support for smarter cache management if/when
>         IndexReader.reopen is added (merging of cached data from subReaders).
> 2) Provide backwards compatibility to support existing FieldCache API with
>     the new implementation, so there is no redundent caching as client code
>     migrades to new API.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org