lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mark Miller (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-831) Complete overhaul of FieldCache API/Implementation
Date Sun, 19 Apr 2009 12:32:48 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12700583#action_12700583
] 

Mark Miller commented on LUCENE-831:
------------------------------------

bq. >>I was also thinking that some of these issues could force back up to multi-reader
support though. 

bq.>Hopefully not...

Yes, I don't know enough yet to know for sure. My thought was things like norms and deletes
that are available from multireader now will have to either still be, or straddle multi/segment
for a while. I guess that doesnt become much of an issue if we go with the same method of
just don't load from both single and multi or you will double your reqs? It just gets ugly
trying to prevent multireader use with valuesource, but then have to support it due to all
the back compat reqs.

bq. We can (and I think should) do FieldType without forcing a fixed schema.

Fair enough, fair enough. I wasn't really taking this completely from this discussion, but
from a variety of ideas about fields that have been spilling out on the list. Of course we
can still get a lot better (easier) without hitting fixed.

bq. For tags we'd presumably want multi-valued fields handled in ValueSource, plus updatability,
plus NRT.

Well I'm glad its a small order. Yonik did do some multi value faceting work that I never
really looked at. I'll go dig it up.

It may just be best if this sits for a while and we see what happens with a couple other issues
floating around it. I said I had sweat to pump into this, not intelligence ;) If we hit all
this stuff (and yes, your not saying we need to or should, but) this ends up touching most
things in IndexReader, than possibly writing and merging and what not in IndexWriter (pluggable
norms etc still need to be written, merged, loaded, etc), and ... 



> Complete overhaul of FieldCache API/Implementation
> --------------------------------------------------
>
>                 Key: LUCENE-831
>                 URL: https://issues.apache.org/jira/browse/LUCENE-831
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>            Reporter: Hoss Man
>            Assignee: Mark Miller
>             Fix For: 3.0
>
>         Attachments: ExtendedDocument.java, fieldcache-overhaul.032208.diff, fieldcache-overhaul.diff,
fieldcache-overhaul.diff, LUCENE-831-trieimpl.patch, LUCENE-831.03.28.2008.diff, LUCENE-831.03.30.2008.diff,
LUCENE-831.03.31.2008.diff, LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch,
LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch,
LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch
>
>
> Motivation:
> 1) Complete overhaul the API/implementation of "FieldCache" type things...
>     a) eliminate global static map keyed on IndexReader (thus
>         eliminating synch block between completley independent IndexReaders)
>     b) allow more customization of cache management (ie: use 
>         expiration/replacement strategies, disk backed caches, etc)
>     c) allow people to define custom cache data logic (ie: custom
>         parsers, complex datatypes, etc... anything tied to a reader)
>     d) allow people to inspect what's in a cache (list of CacheKeys) for
>         an IndexReader so a new IndexReader can be likewise warmed. 
>     e) Lend support for smarter cache management if/when
>         IndexReader.reopen is added (merging of cached data from subReaders).
> 2) Provide backwards compatibility to support existing FieldCache API with
>     the new implementation, so there is no redundent caching as client code
>     migrades to new API.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message