lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <>
Subject [jira] Commented: (LUCENE-831) Complete overhaul of FieldCache API/Implementation
Date Thu, 09 Apr 2009 15:43:12 GMT


Michael McCandless commented on LUCENE-831:

I guess you meant dangerous as in dangerous to reopen then? I actually thought you meant as
in dangerous because it could require too may resources. Dangerous is a tough to pin down
Dangerous is a dangerous word ;)

I meant: I don't like exposing non-performant APIs; they are sneaky traps.  (EG TermEnum.skipTo
is another such API).

bq. So what are the advantages of the iterator API again then?

The big advantage is the possibility of backing it with eg an IndexInput, so that the values
need to all be in RAM for one segment.  Though, as Lucy is doing, we could leave things on
disk and still have random access via mmap, which perhaps should be an option for Lucene as
well.  However, iterator only messes up out-of-order scoring (BooleanScorer for OR queries),
so I'm tentatively leaning against iterator only at this point.

bq. but I'd love to put a little sweat into this issue.

That would be AWESOME (if you can somehow make time)!

We should hash out the design a bit before figuring out how/where to start.

> Complete overhaul of FieldCache API/Implementation
> --------------------------------------------------
>                 Key: LUCENE-831
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>            Reporter: Hoss Man
>             Fix For: 3.0
>         Attachments:, fieldcache-overhaul.032208.diff, fieldcache-overhaul.diff,
fieldcache-overhaul.diff, LUCENE-831.03.28.2008.diff, LUCENE-831.03.30.2008.diff, LUCENE-831.03.31.2008.diff,
LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch,
> Motivation:
> 1) Complete overhaul the API/implementation of "FieldCache" type things...
>     a) eliminate global static map keyed on IndexReader (thus
>         eliminating synch block between completley independent IndexReaders)
>     b) allow more customization of cache management (ie: use 
>         expiration/replacement strategies, disk backed caches, etc)
>     c) allow people to define custom cache data logic (ie: custom
>         parsers, complex datatypes, etc... anything tied to a reader)
>     d) allow people to inspect what's in a cache (list of CacheKeys) for
>         an IndexReader so a new IndexReader can be likewise warmed. 
>     e) Lend support for smarter cache management if/when
>         IndexReader.reopen is added (merging of cached data from subReaders).
> 2) Provide backwards compatibility to support existing FieldCache API with
>     the new implementation, so there is no redundent caching as client code
>     migrades to new API.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message