lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Male (JIRA)" <>
Subject [jira] [Commented] (LUCENE-3312) Break out StorableField from IndexableField
Date Fri, 31 Aug 2012 11:15:09 GMT


Chris Male commented on LUCENE-3312:

I've thought about this a little bit.

To me storing needs no 'type' information at all: But I guess the problem with that is that
we need
DocValues types since DocValues are "stored fields" here.

We've gone back and forwards about this a lot since the Fields cleanup began but it would
be nice to actually have the DocValues Types on the StorableField itself rather than on StorableFieldType.
 In the end the type is related to the type of the value itself, not disconnected metadata.
 Having it this way would also alleviate the need for StorableFieldType and make storing values
as simple as possible.

This basically is the same problem all over again.
* You make a Document with N StorableFields
* You call IR.document and get a StorableDocument back, with N-3 StorableFields.
* You wonder: what happened to the other 3 fields?

They were DocValues.

What if they were returned? Because you're absolutely right, it seems odd for DocValues Fields
to be StorableFields and then not accessible like all other StorableFields.  So what if we
changed how IR.document worked so you could pull DocValues Fields too.  Is that something
users might want?
> Break out StorableField from IndexableField
> -------------------------------------------
>                 Key: LUCENE-3312
>                 URL:
>             Project: Lucene - Core
>          Issue Type: Improvement
>          Components: core/index
>            Reporter: Michael McCandless
>            Assignee: Nikola Tankovic
>              Labels: gsoc2012, lucene-gsoc-12
>             Fix For: Field Type branch
>         Attachments: LUCENE-3312-DocumentIterators-uwe.patch, lucene-3312-patch-01.patch,
lucene-3312-patch-02.patch, lucene-3312-patch-03.patch, lucene-3312-patch-04.patch, lucene-3312-patch-05.patch,
lucene-3312-patch-06.patch, lucene-3312-patch-07.patch, lucene-3312-patch-08.patch, lucene-3312-patch-09.patch,
lucene-3312-patch-10.patch, lucene-3312-patch-11.patch, lucene-3312-patch-12a.patch, lucene-3312-patch-12.patch,
lucene-3312-patch-13.patch, lucene-3312-patch-14.patch, LUCENE-3312-reintegration.patch
> In the field type branch we have strongly decoupled
> Document/Field/FieldType impl from the indexer, by having only a
> narrow API (IndexableField) passed to IndexWriter.  This frees apps up
> use their own "documents" instead of the "user-space" impls we provide
> in oal.document.
> Similarly, with LUCENE-3309, we've done the same thing on the
> doc/field retrieval side (from IndexReader), with the
> StoredFieldsVisitor.
> But, maybe we should break out StorableField from IndexableField,
> such that when you index a doc you provide two Iterables -- one for the
> IndexableFields and one for the StorableFields.  Either can be null.
> One downside is possible perf hit for fields that are both indexed &
> stored (ie, we visit them twice, lookup their name in a hash twice,
> etc.).  But the upside is a cleaner separation of concerns in API....

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message