couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mathias Meyer (JIRA)" <j...@apache.org>
Subject [jira] Updated: (COUCHDB-834) startkey_docid/endkey_docid don't work without an exact startkey/endkey match
Date Mon, 26 Jul 2010 09:29:52 GMT

     [ https://issues.apache.org/jira/browse/COUCHDB-834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Mathias Meyer updated COUCHDB-834:
----------------------------------

    Description: 
This issue popped up when I wanted to paginate through a list of documents using a combined
array key, using a startkey and endkey that's based solely on the first part of said key.
First part is a reference to a different document, second part is a timestamp to keep the
list sorted by creation time. The list of documents can be fetched using startkey=["key"]
and endkey=["key", {}]

Now, I wanted to add pagination to this list, only fetching so many documents starting at
startkey_docid, which failed using this setup. It seems (and Jan validated that assumption
by analyzing the source) that both startkey needs to be an exact match for startkey_docid
to have any effect. If there's no exact match, CouchDB will silently ignore the startkey_docid,
a behaviour that's undocumented and to be quite frank, unintuitive.

Consider the following two documents, both pointing to the same other_id:
{"_id": "one", "other_id": "other", "second_key": "one"}
{"_id": "two", "other_id": "other", "second_key": "two"}

And a simple map/reduce function that just emits the combined key:

{
   "other_documents": {
       "reduce": "_sum",
       "map": "          function(doc) { \n emit([doc.other_id, doc.second_key], 1);\n  }\n"
   }
}

Querying the view like this gives the expected results:

curl 'http://localhost:5984/startkey_bug/_design/other_documents/_view/other_documents?reduce=false&startkey=\["other"\]&endkey=\["other",\{\}\]'

{"total_rows":2,"offset":0,"rows":[
{"id":"one","key":["other","one"],"value":1},
{"id":"two","key":["other","two"],"value":1}
]}

If I add in a startkey_docid of two, I'd expect CouchDB to skip to the second result in the
list, skipping the first, but it doesn't:

curl 'http://localhost:5984/startkey_bug/_design/other_documents/_view/other_documents?reduce=false&startkey=\["other"\]&endkey=\["other",\{\}\]&startkey_docid=two'

{"total_rows":2,"offset":0,"rows":[
{"id":"one","key":["other","one"],"value":1},
{"id":"two","key":["other","two"],"value":1}
]}

However, it does what I'd expect when I specify an exact startkey (the endkey is still the
same):

curl 'http://localhost:5984/startkey_bug/_design/other_documents/_view/other_documents?reduce=false&startkey=\["other","one"\]&endkey=\["other",\{\}\]&startkey_docid=two'

{"total_rows":2,"offset":1,"rows":[
{"id":"two","key":["other","two"],"value":1}
]}

If you add in an exact endkey, the situation doesn't change, and the result is as expected.

Having an exact startkey is an acceptable workaround, but I'd still say this behaviour is
not intuitive, and either should be fixed to work the same in all of the above situations.
If not, at least the documentation should properly reflect these situation, explaining the
proper workarounds.

Update: I just checked how this works out when using descending=true, the same is true for
the swapped endkey and startkey parameters. Specifying and endkey_docid requires to specify
an exact endkey match.

  was:
This issue popped up when I wanted to paginate through a list of documents using a combined
array key, using a startkey and endkey that's based solely on the first part of said key.
First part is a reference to a different document, second part is a timestamp to keep the
list sorted by creation time. The list of documents can be fetched using startkey=["key"]
and endkey=["key", {}]

Now, I wanted to add pagination to this list, only fetching so many documents starting at
startkey_docid, which failed using this setup. It seems (and Jan validated that assumption
by analyzing the source) that both startkey needs to be an exact match for startkey_docid
to have any effect. If there's no exact match, CouchDB will silently ignore the startkey_docid,
a behaviour that's undocumented and to be quite frank, unintuitive.

Consider the following two documents, both pointing to the same other_id:
{"_id": "one", "other_id": "other", "second_key": "one"}
{"_id": "two", "other_id": "other", "second_key": "two"}

And a simple map/reduce function that just emits the combined key:

{
   "other_documents": {
       "reduce": "_sum",
       "map": "          function(doc) { \n emit([doc.other_id, doc.second_key], 1);\n  }\n"
   }
}

Querying the view like this gives the expected results:

curl 'http://localhost:5984/startkey_bug/_design/other_documents/_view/other_documents?reduce=false&startkey=\["other"\]&endkey=\["other",\{\}\]'

{"total_rows":2,"offset":0,"rows":[
{"id":"one","key":["other","one"],"value":1},
{"id":"two","key":["other","two"],"value":1}
]}

If I add in a startkey_docid of two, I'd expect CouchDB to skip to the second result in the
list, skipping the first, but it doesn't:

curl 'http://localhost:5984/startkey_bug/_design/other_documents/_view/other_documents?reduce=false&startkey=\["other"\]&endkey=\["other",\{\}\]&startkey_docid=two'

{"total_rows":2,"offset":0,"rows":[
{"id":"one","key":["other","one"],"value":1},
{"id":"two","key":["other","two"],"value":1}
]}

However, it does what I'd expect when I specify an exact startkey (the endkey is still the
same):

curl 'http://localhost:5984/startkey_bug/_design/other_documents/_view/other_documents?reduce=false&startkey=\["other","one"\]&endkey=\["other",\{\}\]&startkey_docid=two'

{"total_rows":2,"offset":1,"rows":[
{"id":"two","key":["other","two"],"value":1}
]}

If you add in an exact endkey, the situation doesn't change, and the result is as expected.

Having an exact startkey is an acceptable workaround, but I'd still say this behaviour is
not intuitive, and either should be fixed to work the same in all of the above situations.
If not, at least the documentation should properly reflect these situation, explaining the
proper workarounds.


> startkey_docid/endkey_docid don't work without an exact startkey/endkey match
> -----------------------------------------------------------------------------
>
>                 Key: COUCHDB-834
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-834
>             Project: CouchDB
>          Issue Type: Bug
>          Components: JavaScript View Server
>    Affects Versions: 1.0
>            Reporter: Mathias Meyer
>
> This issue popped up when I wanted to paginate through a list of documents using a combined
array key, using a startkey and endkey that's based solely on the first part of said key.
First part is a reference to a different document, second part is a timestamp to keep the
list sorted by creation time. The list of documents can be fetched using startkey=["key"]
and endkey=["key", {}]
> Now, I wanted to add pagination to this list, only fetching so many documents starting
at startkey_docid, which failed using this setup. It seems (and Jan validated that assumption
by analyzing the source) that both startkey needs to be an exact match for startkey_docid
to have any effect. If there's no exact match, CouchDB will silently ignore the startkey_docid,
a behaviour that's undocumented and to be quite frank, unintuitive.
> Consider the following two documents, both pointing to the same other_id:
> {"_id": "one", "other_id": "other", "second_key": "one"}
> {"_id": "two", "other_id": "other", "second_key": "two"}
> And a simple map/reduce function that just emits the combined key:
> {
>    "other_documents": {
>        "reduce": "_sum",
>        "map": "          function(doc) { \n emit([doc.other_id, doc.second_key], 1);\n
 }\n"
>    }
> }
> Querying the view like this gives the expected results:
> curl 'http://localhost:5984/startkey_bug/_design/other_documents/_view/other_documents?reduce=false&startkey=\["other"\]&endkey=\["other",\{\}\]'
> {"total_rows":2,"offset":0,"rows":[
> {"id":"one","key":["other","one"],"value":1},
> {"id":"two","key":["other","two"],"value":1}
> ]}
> If I add in a startkey_docid of two, I'd expect CouchDB to skip to the second result
in the list, skipping the first, but it doesn't:
> curl 'http://localhost:5984/startkey_bug/_design/other_documents/_view/other_documents?reduce=false&startkey=\["other"\]&endkey=\["other",\{\}\]&startkey_docid=two'
> {"total_rows":2,"offset":0,"rows":[
> {"id":"one","key":["other","one"],"value":1},
> {"id":"two","key":["other","two"],"value":1}
> ]}
> However, it does what I'd expect when I specify an exact startkey (the endkey is still
the same):
> curl 'http://localhost:5984/startkey_bug/_design/other_documents/_view/other_documents?reduce=false&startkey=\["other","one"\]&endkey=\["other",\{\}\]&startkey_docid=two'
> {"total_rows":2,"offset":1,"rows":[
> {"id":"two","key":["other","two"],"value":1}
> ]}
> If you add in an exact endkey, the situation doesn't change, and the result is as expected.
> Having an exact startkey is an acceptable workaround, but I'd still say this behaviour
is not intuitive, and either should be fixed to work the same in all of the above situations.
If not, at least the documentation should properly reflect these situation, explaining the
proper workarounds.
> Update: I just checked how this works out when using descending=true, the same is true
for the swapped endkey and startkey parameters. Specifying and endkey_docid requires to specify
an exact endkey match.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message