lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ishan Chattopadhyaya (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SOLR-8220) Read field from docValues for non stored fields
Date Thu, 19 Nov 2015 11:16:11 GMT

    [ https://issues.apache.org/jira/browse/SOLR-8220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15013356#comment-15013356
] 

Ishan Chattopadhyaya edited comment on SOLR-8220 at 11/19/15 11:15 AM:
-----------------------------------------------------------------------

Updated the patch with a minor one line fix in the multi-valued string field case:
{noformat}
             doc.add(schemaField.getType().createField(schemaField, values.lookupOrd(i).utf8ToString(),
1f));
{noformat}

Fyi, since I had already copied over the multivalued fields support from SOLR-8276 patch to
this patch earlier, I've made SOLR-8276 depend on this issue. So, if we fix this issue now,
we'll be able to fix (a) search results with non stored docvalues, (b) RTG of documents containing
non stored docvalues, (c) atomic updates of documents containing non stored docvalues (for
updates to both regular fields as well as non stored docvalues). I will make SOLR-5944 depend
on this now, and update the patch there.


was (Author: ichattopadhyaya):
Updated the patch with a minor one line fix in the multi-valued string field case:
{{                    doc.add(schemaField.getType().createField(schemaField, values.lookupOrd(i).utf8ToString(),
1f));}}

Fyi, since I had already copied over the multivalued fields support from SOLR-8276 patch to
this patch earlier, I've made SOLR-8276 depend on this issue. So, if we fix this issue now,
we'll be able to fix (a) search results with non stored docvalues, (b) RTG of documents containing
non stored docvalues, (c) atomic updates of documents containing non stored docvalues (for
updates to both regular fields as well as non stored docvalues). I will make SOLR-5944 depend
on this now.

> Read field from docValues for non stored fields
> -----------------------------------------------
>
>                 Key: SOLR-8220
>                 URL: https://issues.apache.org/jira/browse/SOLR-8220
>             Project: Solr
>          Issue Type: Improvement
>            Reporter: Keith Laban
>         Attachments: SOLR-8220-ishan.patch, SOLR-8220-ishan.patch, SOLR-8220-ishan.patch,
SOLR-8220-ishan.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch,
SOLR-8220.patch
>
>
> Many times a value will be both stored="true" and docValues="true" which requires redundant
data to be stored on disk. Since reading from docValues is both efficient and a common practice
(facets, analytics, streaming, etc), reading values from docValues when a stored version of
the field does not exist would be a valuable disk usage optimization.
> The only caveat with this that I can see would be for multiValued fields as they would
always be returned sorted in the docValues approach. I believe this is a fair compromise.
> I've done a rough implementation for this as a field transform, but I think it should
live closer to where stored fields are loaded in the SolrIndexSearcher.
> Two open questions/observations:
> 1) There doesn't seem to be a standard way to read values for docValues, facets, analytics,
streaming, etc, all seem to be doing their own ways, perhaps some of this logic should be
centralized.
> 2) What will the API behavior be? (Below is my proposed implementation)
> Parameters for fl:
> - fl="docValueField"
>   -- return field from docValue if the field is not stored and in docValues, if the field
is stored return it from stored fields
> - fl="*"
>   -- return only stored fields
> - fl="+"
>    -- return stored fields and docValue fields
> 2a - would be easiest implementation and might be sufficient for a first pass. 2b - is
current behavior



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message