couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Anderson" <jch...@grabb.it>
Subject Re: Iterating over/paging through large sets of documents
Date Fri, 25 Jul 2008 23:13:04 GMT
John,

I wrote a simple algorithm to page through all the results of a view,
yielding them one key (with all it's values) at a time. I also have a
simple loop for iterating through all_docs. You can see both of them
here:

http://github.com/jchris/couchrest/tree/master/lib/pager.rb

I'm sure I could generalize the all_docs pager to work for any view,
but for now it works for me.

Using "count" is fine when you want to limit a set - it's just using
"skip" that is discouraged.

Chris

On Fri, Jul 25, 2008 at 1:43 PM, John Evans <john@jpevans.com> wrote:
> I think I read somewhere on the wiki that it's better to use startkey/endkey
> to grab "pages" of documents, as opposed to using "count" because "count"
> still has to materialize the entire set of results before filtering to the
> specified count (or something like that).
>
> In a case where you have a reasonable set of start/end key pairs in advance,
> this makes perfect sense, but what is the best practice for paging through a
> set of documents where you don't know any keys?  For example, can can you
> start with count=10 to get the first ten results and then set the startkey
> equal to the key of the 10th result from the first request and use count=11
> to get the next 10 documents (ignoring the first one that overlaps with the
> last set) or will this run into the same problem?
>
> It feels like I am making this more complicated than it needs to be...
> please tell me this is the case :)
>
> Thanks,
> -
> John
>



-- 
Chris Anderson
http://jchris.mfdz.com

Mime
View raw message