incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Barnes <mrtr...@gmail.com>
Subject Re: Iterating all documents
Date Thu, 07 Jul 2011 00:53:22 GMT
Alternatively, how about you page through _all_docs?

1. Query http://server:5984/dbname/_all_docs?limit=100

2. Process that set. Store the id of the last document.

3. Query 
http://server:5984/dbname/_all_docs?limit=100&startkey=(last_id)&skip=1 
to get the next set.

4. Repeat 2 and 3 until the returned set is empty.

If your batch processing has to be able to resume after being 
terminated, just store the last_id in a file between each set.

If you documents come from a specific view, you can do that too, the 
only difference would be that 'startkey' needs to be the last record's 
view key, and you may also need a 'startkey_docid=(last_id)' parameter 
if the keys are not unique.

-Patrick

On 7/07/2011 9:41 AM, Matthias Eck wrote:
> Hello,
>
> I need to add a new field to all documents in my database.
>
> To have a better control I wanted to do this by batches and defined 2 views:
> documents_without_newfield
> documents_with_newfield
>
> My idea was to just take the first 100 returned by the
> documents_without_newfield view, calculate the new field for all of
> them, save them and take the next 100 etc.
>
> As it turns out the views do not seem to be updated immediately, which
> means that the view documents_without_newfield returns about 95
> documents that actually already had the new field calculated in the
> previous step.
>
> Can I force the view to update immediately so I can iterate through
> the all documents?
>
> Thanks,
> Matthias
>

Mime
View raw message