couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Adam Kocoloski (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (COUCHDB-1797) Query against a view w/ include_docs=true yields doc:null
Date Fri, 08 Nov 2013 16:07:31 GMT

    [ https://issues.apache.org/jira/browse/COUCHDB-1797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13817387#comment-13817387
] 

Adam Kocoloski commented on COUCHDB-1797:
-----------------------------------------

I think that patch has too much of a performance impact to merge (it's a gen_server:call per
view row), but I do see a race condition here that could lead to these {not_found, missing}
/ null results.

If you look at the code where we obtain the view group state:

https://github.com/apache/couchdb/blob/1.5.0/src/couch_mrview/src/couch_mrview.erl#L75

you can see that we already have the Db open.  couch_mrivew ensures that the index state we
return is at a sequence >= the sequence of the Db:

https://github.com/apache/couchdb/blob/1.5.0/src/couch_mrview/src/couch_mrview_util.erl#L41-L50

It's quite possible that couch_index had to fire off an updater to satisfy that condition,
and when the updater starts it gets the latest Db:

https://github.com/apache/couchdb/blob/1.5.0/src/couch_index/src/couch_index_updater.erl#L129

If updates landed in between opening the Db used by query_view and opening the Db used by
couch_index_updater then we can end up with updates in the view index that are not yet accessible
by the Db held by query_view.  Attempts to open these latest docs will then come back with
{not_found, missing}.

Re-opening the Db in query_view after we receive the index state but before we start the fold
alleviates the performance impact, but it's still a race condition.  The {not_found, missing}
errors will disappear, but in their place we might end up with {not_found, deleted} errors
or documents that are at a newer revision than the revision that was included in the index.

A really strictly consistent solution would have to find a way to communicate the exact #db
used by the indexer back to the query handler.  That's non-trivial to say the least.

I'm not opposed to a single reopen in the query handler after we receive the index state (maybe
we can make it conditional on the sequence of the secondary index being greater than the sequence
of the original DB), but we should be clear that it really just trades one inconsistency for
another.

> Query against a view w/ include_docs=true yields doc:null
> ---------------------------------------------------------
>
>                 Key: COUCHDB-1797
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-1797
>             Project: CouchDB
>          Issue Type: Bug
>          Components: Database Core
>    Affects Versions: 1.3
>            Reporter: Marten Feldtmann
>            Priority: Critical
>         Attachments: 0001-fix-null-doc-or-doc-not-found-when-including-it-in-a.patch
>
>
> I have a test system (CouchDB 1.3.0 under windows 7/64 Bit), where I insert lots of documents(100000)
to get a regular flow of new incoming documents.
> The database has also some views (25). Another task is reading documents from a view,
change some attributes and then the document should disappear from the view (and appear in
another view). Something like a state machine.
> I do the following query against a view:
> http://localhost:5984/test5/_design/ticketorggraphics/_view/waiting?limit=5&include_docs=true&include_end=true
> and get the following string:
> {"total_rows":9224,"offset":0,"rows":[
> {"id":"92224c2862ef49c15f3d51e3e7283cde","key":"ni13_1_2013101_20130518155518065","value":"ni13_2013101_VsPercentResult_83519204509647","doc":{"_id":"92224c2862ef49c15f3d51e3e7283cde","_rev":"1-6e8b57744c2b88f3d2dcad602e2e3353","distribution":"global","type":"ESTicketCreateOriginalGraphic","createdTS":"20130518155518065","priority":1,"eDayID":"ni13","electionID":2013101,"layout":"ard2013","baseDocKey":"ni13_2013101_VsPercentResult_83519204509647","_attachments":{"bclitemsdata":{"content_type":"application/zip","revpos":1,"digest":"md5-sdff2jkxp4w6xsSlx5Yzxw==","length":4533,"stub":true}}}},
> {"id":"92224c2862ef49c15f3d51e3e7285d38","key":"ni13_1_2013101_20130518155518135","value":"ni13_2013101_VsPercentResult_16771681663808","doc":{"_id":"92224c2862ef49c15f3d51e3e7285d38","_rev":"1-05b92282ec5f2d1b6d56912760792702","distribution":"global","type":"ESTicketCreateOriginalGraphic","createdTS":"20130518155518135","priority":1,"eDayID":"ni13","electionID":2013101,"layout":"ard2013","baseDocKey":"ni13_2013101_VsPercentResult_16771681663808","_attachments":{"bclitemsdata":{"content_type":"application/zip","revpos":1,"digest":"md5-sdff2jkxp4w6xsSlx5Yzxw==","length":4533,"stub":true}}}},
> {"id":"92224c2862ef49c15f3d51e3e728c5e2","key":"ni13_1_2013101_20130518155518413","value":"ni13_2013101_VsPercentResult_10619936276742","doc":null},
> {"id":"b02d128d88188d78ef7f3478586c349b","key":"ni13_2_2013101_20130518155203924","value":"ni13_2013101_VsPercentResult_46202583943140","doc":{"_id":"b02d128d88188d78ef7f3478586c349b","_rev":"1-bd2a2bfb1f7461e6b3ec6d4d106e08e4","distribution":"global","type":"ESTicketCreateOriginalGraphic","createdTS":"20130518155203924","priority":2,"eDayID":"ni13","electionID":2013101,"layout":"ard2013","baseDocKey":"ni13_2013101_VsPercentResult_46202583943140","_attachments":{"bclitemsdata":{"content_type":"application/zip","revpos":1,"digest":"md5-ErSTxuEaSHZrIJ276ANdOQ==","length":4533,"stub":true}}}},
> {"id":"b02d128d88188d78ef7f3478586c72a1","key":"ni13_2_2013101_20130518155204124","value":"ni13_2013101_VsPercentResult_91455901404013","doc":{"_id":"b02d128d88188d78ef7f3478586c72a1","_rev":"1-6f702cff1d12f992d745fc19be84b846","distribution":"global","type":"ESTicketCreateOriginalGraphic","createdTS":"20130518155204124","priority":2,"eDayID":"ni13","electionID":2013101,"layout":"ard2013","baseDocKey":"ni13_2013101_VsPercentResult_91455901404013","_attachments":{"bclitemsdata":{"content_type":"application/zip","revpos":1,"digest":"md5-L9UXpNNRUyk/LSlzAyEsXQ==","length":4533,"stub":true}}}}
> ]}
> Please notice, that the doc attribute of the third item in the rows attribute has the
value "null". That means, that the view entry has no belonging document ????????
> This seems to indicate an inconsistency within the database.
> I can not say, when this error happens, but I know, that it will happen in my test run.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message