lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hoss Man (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-5944) Support updates of numeric DocValues
Date Wed, 07 Dec 2016 05:05:58 GMT

    [ https://issues.apache.org/jira/browse/SOLR-5944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15727716#comment-15727716
] 

Hoss Man commented on SOLR-5944:
--------------------------------

bq. I saw consistent failures on...

I'm seeing consistent failures from most of the randomized tests.

bq. I haven't looked deep into why it could be failing, but a preliminary look at the logs
lead me to believe that it is a test problem.

can you elaborate on what in the logs gave you that impression?

If it were a test bug -- ie: a bug in tracking the model state compared to the inplace atomic
updes -- I would expect the failures to reproduce if you switched the test to use a regular
(indexed+stored) long field instead of a DVO field -- ie: use the classic atomic update code
instead of the inplace update code.

But when i tried toggling the field used (see comments in {{checkRandomReplay}}) I couldn't
reproduce any failures.

I added some hackish logging to {{checkRandomReplay}} to get it to dump a short sequence that
failed and turned that into a new test method ({{testReplay_nocommit}}) and then i distilled
what seems to be the key problematic bits into an even shorter test: {{testReplay_SetOverriddenWithNoValueThenInc}}
...

{code}
  public void testReplay_SetOverriddenWithNoValueThenInc() throws Exception {
    final String inplaceField = "inplace_l_dvo"; 
    // final String inplaceField = "inplace_nocommit_not_really_l"; // nocommit: "inplace_l_dvo"
    
    checkReplay(inplaceField,
                //
                sdoc("id", "1", inplaceField, map("set", 555L)),
                SOFTCOMMIT,
                sdoc("id", "1", "regular_l", 666L), // NOTE: no inplaceField, regular add
w/overwrite 
                sdoc("id", "1", inplaceField, map("inc", -77)),
                HARDCOMMIT);
  }
{code}

...all of that is now on the branch.

If you toggle the above code to use regular atomic updates, then it passes -- but as written,
so it uses the new inplace update code paths, it fails like so...

{noformat}
   [junit4] FAILURE 0.54s | TestInPlaceUpdatesStandalone.testReplay_SetOverriddenWithNoValueThenInc
<<<
   [junit4]    > Throwable #1: java.lang.AssertionError: expected:<-77> but was:<478>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([9D6E895FCBA28315:6DDD2091B324AFF2]:0)
   [junit4]    > 	at org.apache.solr.update.TestInPlaceUpdatesStandalone.checkReplay(TestInPlaceUpdatesStandalone.java:920)
   [junit4]    > 	at org.apache.solr.update.TestInPlaceUpdatesStandalone.testReplay_SetOverriddenWithNoValueThenInc(TestInPlaceUpdatesStandalone.java:590)
{noformat}

...looks like a genuine bug to me: when a regular update overwrites a doc that had a DVO field
value, a subsequent "inc" operation on the DVO fields is picking up the _old_ value instead
of operating against an implicit default of 0.

(This kind of corner case is what makes randomized testing totally worth the time and effort)

bq. Btw, do you know how to enable commit notifications to show up here for the jira/solr-5944
branch?

IIRC comments about commits to jira/* branches are suppressed intentionally as noise, because
it's expected that there will be lots of iteration on the branches, some of which might be
thrown away, and for posterity what matters is only commits to main line dev branches

> Support updates of numeric DocValues
> ------------------------------------
>
>                 Key: SOLR-5944
>                 URL: https://issues.apache.org/jira/browse/SOLR-5944
>             Project: Solr
>          Issue Type: New Feature
>            Reporter: Ishan Chattopadhyaya
>            Assignee: Shalin Shekhar Mangar
>         Attachments: DUP.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, TestStressInPlaceUpdates.eb044ac71.beast-167-failure.stdout.txt,
TestStressInPlaceUpdates.eb044ac71.beast-587-failure.stdout.txt, TestStressInPlaceUpdates.eb044ac71.failures.tar.gz,
defensive-checks.log.gz, hoss.62D328FA1DEA57FD.fail.txt, hoss.62D328FA1DEA57FD.fail2.txt,
hoss.62D328FA1DEA57FD.fail3.txt, hoss.D768DD9443A98DC.fail.txt, hoss.D768DD9443A98DC.pass.txt
>
>
> LUCENE-5189 introduced support for updates to numeric docvalues. It would be really nice
to have Solr support this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message