couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brad King" <>
Subject Re: Iterating over/paging through large sets of documents
Date Mon, 28 Jul 2008 03:24:43 GMT
I think the iterator mechanisms in couchdb are one of the things that
are underestimated. I've been prototyping an inventory system for a
few weeks, and couchdb views are soo much easier to paginate than,
say, a SQL server DB, because the indexes are explicit, and exposed
directly as views. What I do is simply request a key and a count of +1
more than I need for a page, then use that last value as my start key
for the next page. Simple, clean, fast. I don't even bother trying to
use and end key. Its not feasible for most applications of paging.

On Fri, Jul 25, 2008 at 7:13 PM, Chris Anderson <> wrote:
> John,
> I wrote a simple algorithm to page through all the results of a view,
> yielding them one key (with all it's values) at a time. I also have a
> simple loop for iterating through all_docs. You can see both of them
> here:
> I'm sure I could generalize the all_docs pager to work for any view,
> but for now it works for me.
> Using "count" is fine when you want to limit a set - it's just using
> "skip" that is discouraged.
> Chris
> On Fri, Jul 25, 2008 at 1:43 PM, John Evans <> wrote:
>> I think I read somewhere on the wiki that it's better to use startkey/endkey
>> to grab "pages" of documents, as opposed to using "count" because "count"
>> still has to materialize the entire set of results before filtering to the
>> specified count (or something like that).
>> In a case where you have a reasonable set of start/end key pairs in advance,
>> this makes perfect sense, but what is the best practice for paging through a
>> set of documents where you don't know any keys?  For example, can can you
>> start with count=10 to get the first ten results and then set the startkey
>> equal to the key of the 10th result from the first request and use count=11
>> to get the next 10 documents (ignoring the first one that overlaps with the
>> last set) or will this run into the same problem?
>> It feels like I am making this more complicated than it needs to be...
>> please tell me this is the case :)
>> Thanks,
>> -
>> John
> --
> Chris Anderson

View raw message