lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hoss Man (JIRA)" <>
Subject [jira] Updated: (LUCENE-831) Complete overhaul of FieldCache API/Implementation
Date Wed, 14 Mar 2007 07:28:09 GMT


Hoss Man updated LUCENE-831:

    Attachment: fieldcache-overhaul.diff

This is based on an idea i had a few months ago and was recently reminded of because of several
mail threads about FieldCache .. so i started fleshing it out on the plane last week.

I'm not entirely happy with it in it's current state, but I wanted to post it and see what
people think of the overall approach.

if people like the direction this is going, I would definitely appreciate some help with API
critique and good unit tests (so far i've been relying solely on the existing Unit Tests to
validate that i'm not breaking anything -- but that doesn't really prove that the new APIs
work the way they are intended to)


 * lots of little :TODO:s in code, mainly about javadocs
 * add merge methods to StringIndexCacheKey 
   (a bit complicated, but should be possible/efficient)
 * figure out if there is any better way of dealing with 
   SortComparatorCacheKey and the visibility issues of 
 * change norm caching to use new caches (if not the same 
   main cache, then at least the same classes in a private cache)
 * write an ass load more tests
 * is there a better way to deal with merging then to pass starts[] ?  
   (pass a new datastructure encapsulating starts/subReaders?)
 * CacheFactory seemed like a good idea initially so that MultiReader 
   on a multi-segment index could cascade down, but what if people 
   only want caching at the outermost level (regardless of wether 
   the key is mergable) ... factory can't assuming anything if reader 
   is not an instance of MultiReader
 * audit/change all core code using FieldCache to use new API
 * performance test that this doesn't hurt things in some way.

> Complete overhaul of FieldCache API/Implementation
> --------------------------------------------------
>                 Key: LUCENE-831
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>            Reporter: Hoss Man
>         Attachments: fieldcache-overhaul.diff
> Motivation:
> 1) Complete overhaul the API/implementation of "FieldCache" type things...
>     a) eliminate global static map keyed on IndexReader (thus
>         eliminating synch block between completley independent IndexReaders)
>     b) allow more customization of cache management (ie: use 
>         expiration/replacement strategies, disk backed caches, etc)
>     c) allow people to define custom cache data logic (ie: custom
>         parsers, complex datatypes, etc... anything tied to a reader)
>     d) allow people to inspect what's in a cache (list of CacheKeys) for
>         an IndexReader so a new IndexReader can be likewise warmed. 
>     e) Lend support for smarter cache management if/when
>         IndexReader.reopen is added (merging of cached data from subReaders).
> 2) Provide backwards compatibility to support existing FieldCache API with
>     the new implementation, so there is no redundent caching as client code
>     migrades to new API.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message