lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Earwin Burrfoot (JIRA)" <>
Subject [jira] Commented: (LUCENE-831) Complete overhaul of FieldCache API/Implementation
Date Sun, 12 Apr 2009 17:54:14 GMT


Earwin Burrfoot commented on LUCENE-831:

bq. I'm kind of worried that any change is going to hurt Apps like Solr - if you end up using
the new built in API, but also have code that must stick for a while with the old API (for
multireader fieldcache or something), you'll likely increase RAM usage more than before -
how much of a concern that ends up being, I'm not sure.
My personal stance is that until you have one perfectly thought out API, nothing should restrain
you from changing it. It's better to feel pain once or twice, when you adapt to API changes,
then to feel it constantly, each time you're using that half-assed thing you're keeping around
for back-compat. Look at google-collections. They did some really breaking changes since they
released, but most of them eased my life after I made my project compile and run with the
new version of their library.

bq. In Lucy, I expect StringType to fill this role. FullTextType is for multi-token fields.
In our case multiplicity is defined on FieldInfo level. Because we can have one Int field
that holds some value, and another Int field that holds several ids.
Same goes for the String field - you might want tags on a document that are represented as
untokenized strings, but each document can have 0..n of them.

> Complete overhaul of FieldCache API/Implementation
> --------------------------------------------------
>                 Key: LUCENE-831
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>            Reporter: Hoss Man
>            Assignee: Mark Miller
>             Fix For: 3.0
>         Attachments:, fieldcache-overhaul.032208.diff, fieldcache-overhaul.diff,
fieldcache-overhaul.diff, LUCENE-831.03.28.2008.diff, LUCENE-831.03.30.2008.diff, LUCENE-831.03.31.2008.diff,
LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch,
LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch
> Motivation:
> 1) Complete overhaul the API/implementation of "FieldCache" type things...
>     a) eliminate global static map keyed on IndexReader (thus
>         eliminating synch block between completley independent IndexReaders)
>     b) allow more customization of cache management (ie: use 
>         expiration/replacement strategies, disk backed caches, etc)
>     c) allow people to define custom cache data logic (ie: custom
>         parsers, complex datatypes, etc... anything tied to a reader)
>     d) allow people to inspect what's in a cache (list of CacheKeys) for
>         an IndexReader so a new IndexReader can be likewise warmed. 
>     e) Lend support for smarter cache management if/when
>         IndexReader.reopen is added (merging of cached data from subReaders).
> 2) Provide backwards compatibility to support existing FieldCache API with
>     the new implementation, so there is no redundent caching as client code
>     migrades to new API.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message