lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hoss Man (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-5944) Support updates of numeric DocValues
Date Fri, 09 Dec 2016 00:22:58 GMT

    [ https://issues.apache.org/jira/browse/SOLR-5944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15733820#comment-15733820
] 

Hoss Man commented on SOLR-5944:
--------------------------------


I've pushed an update to {{TestInPlaceUpdatesDistrib}} that refactors some randomized index
building into a new {{buildRandomIndex}} helper method which is now used by most of the "test"
methods in that class.

It's *NOT* currently used by {{docValuesUpdateTest()}} even though it was designed to be --
I made several aborted attempts to try and switch that method to use {{buildRandomIndex}}
knowing that many assertions in that test would need other tweaks to account for more docs
in the index, but i kept running into weird failures that took me a while to explain.

Ultimately I realized the problem is that currently, {{schema-inplace-updates.xml}} is configured
with {{inplace_updatable_float}} having a {{default="0"}} setting -- which (besides making
most of our testing using hta field much weaker then i realized) means that the initial sanity
checks in {{docValuesUpdateTest()}} are even less useful then i originally thought.

----

[~ichattopadhyaya]: do you remember why this default is set on {{inplace_updatable_float}}
(and {{inplace_updatable_int}}) {{schema-inplace-updates.xml}} ? ... i see {{TestInPlaceUpdatesDistrib}}
doing a preliminary sanity check assertion that the defaults exists in the schema, but I don't
see any test that seems to care/expect that default to work, and it seems to weaken our test
coverage of the more common case...

Specifically: when I tried to remove it, I started seeing NPEs from {{SolrIndexSearcher.decorateDocValueFields}}
in various tests:


{noformat}
   [junit4] ERROR   0.05s J2 | TestInPlaceUpdatesStandalone.testUpdateTwoDifferentFields <<<
   [junit4]    > Throwable #1: java.lang.NullPointerException
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([29D61963E75459C5:26F189B6F032D44A]:0)
   [junit4]    > 	at org.apache.solr.search.SolrIndexSearcher.decorateDocValueFields(SolrIndexSearcher.java:810)
   [junit4]    > 	at org.apache.solr.handler.component.RealTimeGetComponent.getInputDocument(RealTimeGetComponent.java:599)
   [junit4]    > 	at org.apache.solr.update.processor.AtomicUpdateDocumentMerger.doInPlaceUpdateMerge(AtomicUpdateDocumentMerger.java:286)
   [junit4]    > 	at org.apache.solr.update.processor.DistributedUpdateProcessor.getUpdatedDocument(DistributedUpdateProcessor.java:1414)
   [junit4]    > 	at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:1072)
   [junit4]    > 	at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:751)
   [junit4]    > 	at org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processAdd(LogUpdateProcessorF
{noformat}


...while it certainly makes sense to have some testing of inplace updates when there is a
schema specified {{default}} that's *non-zero* (although see the previously mentioned SOLR-9838
for some issues with doing that currently), i'm now concerned about how much of the code may
*only* be working _because_ these fields have an explicit {{default="0"}} ?


> Support updates of numeric DocValues
> ------------------------------------
>
>                 Key: SOLR-5944
>                 URL: https://issues.apache.org/jira/browse/SOLR-5944
>             Project: Solr
>          Issue Type: New Feature
>            Reporter: Ishan Chattopadhyaya
>            Assignee: Shalin Shekhar Mangar
>         Attachments: DUP.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, TestStressInPlaceUpdates.eb044ac71.beast-167-failure.stdout.txt,
TestStressInPlaceUpdates.eb044ac71.beast-587-failure.stdout.txt, TestStressInPlaceUpdates.eb044ac71.failures.tar.gz,
defensive-checks.log.gz, hoss.62D328FA1DEA57FD.fail.txt, hoss.62D328FA1DEA57FD.fail2.txt,
hoss.62D328FA1DEA57FD.fail3.txt, hoss.D768DD9443A98DC.fail.txt, hoss.D768DD9443A98DC.pass.txt
>
>
> LUCENE-5189 introduced support for updates to numeric docvalues. It would be really nice
to have Solr support this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message