lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ishan Chattopadhyaya (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-8220) Read field from docValues for non stored fields
Date Sun, 27 Dec 2015 10:49:49 GMT

    [ https://issues.apache.org/jira/browse/SOLR-8220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15072116#comment-15072116
] 

Ishan Chattopadhyaya commented on SOLR-8220:
--------------------------------------------

I suggest we change:

{code}
+  Also note that returning stored fields from docValues (default in schema versions 1.6+)
returns multiValued
+  fields in sorted order. If you require the older behavior of multiValued fields being returned
in the
+  original insertion order, set useDocValuesAsStored="false" for the individual fields or
make
+  sure your schema version is < 1.6. This does not require re-indexing.
+  See SOLR-8220 for more details.
{code}

to 

{code}
+  Also note that while returning non-stored fields from docValues (default in schema versions
1.6+, unless useDocValuesAsStored is false) returns multiValued
+  fields in sorted order. If you require the multiValued fields being returned in the
+  original insertion order, then make your multiValued field as stored. This requires re-indexing.
+  See SOLR-8220 for more details.
 {code}

> Read field from docValues for non stored fields
> -----------------------------------------------
>
>                 Key: SOLR-8220
>                 URL: https://issues.apache.org/jira/browse/SOLR-8220
>             Project: Solr
>          Issue Type: Improvement
>            Reporter: Keith Laban
>            Assignee: Shalin Shekhar Mangar
>         Attachments: SOLR-8220-5x.patch, SOLR-8220-ishan.patch, SOLR-8220-ishan.patch,
SOLR-8220-ishan.patch, SOLR-8220-ishan.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch,
SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch,
SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch,
SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch,
SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch,
SOLR-8220.patch
>
>
> Many times a value will be both stored="true" and docValues="true" which requires redundant
data to be stored on disk. Since reading from docValues is both efficient and a common practice
(facets, analytics, streaming, etc), reading values from docValues when a stored version of
the field does not exist would be a valuable disk usage optimization.
> The only caveat with this that I can see would be for multiValued fields as they would
always be returned sorted in the docValues approach. I believe this is a fair compromise.
> I've done a rough implementation for this as a field transform, but I think it should
live closer to where stored fields are loaded in the SolrIndexSearcher.
> Two open questions/observations:
> 1) There doesn't seem to be a standard way to read values for docValues, facets, analytics,
streaming, etc, all seem to be doing their own ways, perhaps some of this logic should be
centralized.
> 2) What will the API behavior be? (Below is my proposed implementation)
> Parameters for fl:
> - fl="docValueField"
>   -- return field from docValue if the field is not stored and in docValues, if the field
is stored return it from stored fields
> - fl="*"
>   -- return only stored fields
> - fl="+"
>    -- return stored fields and docValue fields
> 2a - would be easiest implementation and might be sufficient for a first pass. 2b - is
current behavior



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message