lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hoss Man (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-5944) Support updates of numeric DocValues
Date Wed, 07 Dec 2016 19:37:58 GMT

    [ https://issues.apache.org/jira/browse/SOLR-5944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15729698#comment-15729698
] 

Hoss Man commented on SOLR-5944:
--------------------------------

bq. ...looks like a genuine bug to me: ...

FWIW:

I made a half assed attempt to quickly reproduce this with {{bin/solr -e schemaless}} and
some curl commands to create fields and add docs -- and i couldn't reproduce this.

It's possible I screwed something up with the commands, and or varried the input slightly
in a way that didn't tickle the bug; or it's possible that with the default schemaless configs,
some (default?) autocommit setting caued the tlogs to get flushed/committed in a way that
bypasses the bug.


In any case, I then attempted to more systematically demo the bug using the same configs as
the test -- and was able to easily reproduce...

{noformat}
mkdir -p /tmp/solr_test_home/cores/inplace_testing/conf
cp server/solr/solr.xml /tmp/solr_test_home/
touch /tmp/solr_test_home/cores/inplace_testing/core.properties
cp core/src/test-files/solr/collection1/conf/schema-inplace-updates.xml /tmp/solr_test_home/cores/inplace_testing/conf/schema.xml
cp core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml /tmp/solr_test_home/cores/inplace_testing/conf/solrconfig.xml
cp core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml /tmp/solr_test_home/cores/inplace_testing/conf/

# normally these are randomized by the test harness
# i just picked these values arbitrarily since it seemed to reproduce regardless of seed
bin/solr start -f -s /tmp/solr_test_home/ -Dsolr.tests.maxBufferedDocs=1000 -Dsolr.tests.ramBufferSizeMB=100
-Dsolr.tests.mergeScheduler=org.apache.lucene.index.ConcurrentMergeScheduler -Dsolr.tests.useMergePolicy=false
-Dsolr.tests.mergePolicyFactory=org.apache.solr.index.TieredMergePolicyFactory

curl -X POST -H 'Content-Type: application/json' --data-binary '[
{ "id": "1","inplace_l_dvo": { "set": 555 } },
{ "id": "2","not_inplace_l": { "set": 555 } }
]' 'http://localhost:8983/solr/inplace_testing/update?commit=true&softCommit=true'

curl -X POST -H 'Content-Type: application/json' --data-binary '[
{ "id": "1","regular_l": 666 },
{ "id": "2","regular_l": 666 }
]' 'http://localhost:8983/solr/inplace_testing/update'

curl -X POST -H 'Content-Type: application/json' --data-binary '[
{ "id": "1","inplace_l_dvo": { "inc": -77 } },
{ "id": "2","not_inplace_l": { "inc": -77 } }
]' 'http://localhost:8983/solr/inplace_testing/update'

curl 'http://localhost:8983/solr/inplace_testing/get?wt=json&indent=true&ids=1,2'
{noformat}

Final RTG Output...

{noformat}
{
  "response":{"numFound":2,"start":0,"docs":[
      {
        "id":"1",
        "inplace_l_dvo":478,
        "_version_":1553084875040358400,
        "regular_l":666},
      {
        "id":"2",
        "regular_l":666,
        "not_inplace_l":-77,
        "_version_":1553084875042455552,
        "copyfield1_src__both_updateable":0,
        "inplace_updatable_float":0.0,
        "copyfield2_src__only_src_updatable":0,
        "inplace_updatable_int":0}]
  }}
{noformat}

...note that unlike in the test where we only deal with a single document, I used 2 docs here
-- giving them identical updates -- to show that while the problem reproduces when using a
DVO field that gets inplace updates (doc id #1) it doesn't reproduce when using a regular
stored+indexed field that gets a classic atomic update (doc id #2).

----

side note: i was confused by all those fields with a value of "0" in doc id#2, and thought
that might be somehow related to the bug -- ie: DVO fields getting added with the default
0 in some code path -- but then i realized they all have defaultValues in the schema.xml --
i'm not entirely sure why only doc#2 shows them in this RTG, but doc#1 gets those field values
as well once a commit happens.

I suspect there is some code path in the regular atomic update logic where docs read from
the tlog get any default field values added to them, and that isn't happening in the inplace
code path case? ... not certain ... might be another bug to look into?

perhaps the {{// Note: We don't need to add default fields ... would've happened during the
full indexing initially}} isn't necessarily true for something that so far only exists in
the tlog? ... need to thin about this some more.

> Support updates of numeric DocValues
> ------------------------------------
>
>                 Key: SOLR-5944
>                 URL: https://issues.apache.org/jira/browse/SOLR-5944
>             Project: Solr
>          Issue Type: New Feature
>            Reporter: Ishan Chattopadhyaya
>            Assignee: Shalin Shekhar Mangar
>         Attachments: DUP.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, TestStressInPlaceUpdates.eb044ac71.beast-167-failure.stdout.txt,
TestStressInPlaceUpdates.eb044ac71.beast-587-failure.stdout.txt, TestStressInPlaceUpdates.eb044ac71.failures.tar.gz,
defensive-checks.log.gz, hoss.62D328FA1DEA57FD.fail.txt, hoss.62D328FA1DEA57FD.fail2.txt,
hoss.62D328FA1DEA57FD.fail3.txt, hoss.D768DD9443A98DC.fail.txt, hoss.D768DD9443A98DC.pass.txt
>
>
> LUCENE-5189 introduced support for updates to numeric docvalues. It would be really nice
to have Solr support this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message