lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Toke Eskildsen ...@statsbiblioteket.dk>
Subject Re: Is it safe to upgrade an existing field to docvalues?
Date Thu, 25 Aug 2016 10:50:25 GMT
Ronald Wood <rwood@smarsh.com> wrote:
> Did you find you had to do a full conversion all at once because simply turning on
> docvalues in the schema caused issues?

Yes.

> I ask because my presupposition has been that we could turn it on without any
> harm as we incrementally converted our indexes.

If you don't use the field for any queries until all the values has been re-build, I guess
that would work. The I-am-not-so-sure-part is how the merger handles the case of a field having
DocValues in one segment and not in another.

But mixing docValued & non-docValued segments with a schema that says DocValues will make
DocValue-using queries fail, as you seem to have encountered:

> But this won’t work if enabling docvalues in the schema will lead to errors when
> fields don’t have docvalues actually populated. I.e.. the “IllegalStateException:
> unexpected docvalues type NONE for field 'id' (expected=SORTED)” error I see.

> I’m still trying to get to the bottom of whether that error means I cannot safely do
> an incremental conversion in-place.

When you enable docValues in the Solr schema, Solr also uses that information when reading
the data from the segments, so when the code detects the missing docValues in the segments
themselves, it is already too far down the execution path to change strategy. Basically the
contract (schema) is broken, so all bets are off.

Your gradual enabling would work, at least for faceting, if it was possible to force the selection
code to only use the indexed values, but the current code does not have an option for forcing
the use of the indexed value. You could add it as a (small) hack, if you are comfortable with
that. I don't know how easy or hard it would to hack grouping.

- Toke Eskildsen

Mime
View raw message