lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-831) Complete overhaul of FieldCache API/Implementation
Date Sun, 19 Apr 2009 09:55:47 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12700571#action_12700571
] 

Michael McCandless commented on LUCENE-831:
-------------------------------------------

bq. I was also thinking that some of these issues could force back up to multi-reader support
though. 

Hopefully not...

bq. I want field handling to become easier in Lucene, but I hope we don't lose any of our
super on the fly settings. +1 on making field handing easier, but I am much more weary of
a fixed schema type thing.

I think "consolidating per-field details" (FieldType) is well decoupled from "forcing every
occurrence of a field to be the same" (fixed schema).  We can (and I think should) do FieldType
without forcing a fixed schema.

bq. I am very interested in having updatable CSF's (much too easy to mistype that). There
are many cool things to use it for, especially in combination with near realtime search (tagging
variations).

For tags we'd presumably want multi-valued fields handled in ValueSource, plus updatability,
plus NRT.

Updatability is tricky... ValueSource would maybe need a "startChanges()" API, which would
copy the array (copy-on-write) if it's not already private.  The problem is... direct array
access precludes more efficient data structures that amortize the copy-on-write cost (eg by
block), which we are wanting to eventually get to for deleted docs & norms (it's likely
a large cost in NRT reader turnaround, though I hven't yet measured just how costly).

> Complete overhaul of FieldCache API/Implementation
> --------------------------------------------------
>
>                 Key: LUCENE-831
>                 URL: https://issues.apache.org/jira/browse/LUCENE-831
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>            Reporter: Hoss Man
>            Assignee: Mark Miller
>             Fix For: 3.0
>
>         Attachments: ExtendedDocument.java, fieldcache-overhaul.032208.diff, fieldcache-overhaul.diff,
fieldcache-overhaul.diff, LUCENE-831-trieimpl.patch, LUCENE-831.03.28.2008.diff, LUCENE-831.03.30.2008.diff,
LUCENE-831.03.31.2008.diff, LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch,
LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch,
LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch, LUCENE-831.patch
>
>
> Motivation:
> 1) Complete overhaul the API/implementation of "FieldCache" type things...
>     a) eliminate global static map keyed on IndexReader (thus
>         eliminating synch block between completley independent IndexReaders)
>     b) allow more customization of cache management (ie: use 
>         expiration/replacement strategies, disk backed caches, etc)
>     c) allow people to define custom cache data logic (ie: custom
>         parsers, complex datatypes, etc... anything tied to a reader)
>     d) allow people to inspect what's in a cache (list of CacheKeys) for
>         an IndexReader so a new IndexReader can be likewise warmed. 
>     e) Lend support for smarter cache management if/when
>         IndexReader.reopen is added (merging of cached data from subReaders).
> 2) Provide backwards compatibility to support existing FieldCache API with
>     the new implementation, so there is no redundent caching as client code
>     migrades to new API.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message