From user-return-17024-apmail-couchdb-user-archive=couchdb.apache.org@couchdb.apache.org Thu Jul 7 10:17:38 2011 Return-Path: X-Original-To: apmail-couchdb-user-archive@www.apache.org Delivered-To: apmail-couchdb-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 19EFB468B for ; Thu, 7 Jul 2011 10:17:38 +0000 (UTC) Received: (qmail 5411 invoked by uid 500); 7 Jul 2011 10:17:36 -0000 Delivered-To: apmail-couchdb-user-archive@couchdb.apache.org Received: (qmail 5169 invoked by uid 500); 7 Jul 2011 10:17:31 -0000 Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@couchdb.apache.org Delivered-To: mailing list user@couchdb.apache.org Received: (qmail 5161 invoked by uid 99); 7 Jul 2011 10:17:29 -0000 Received: from minotaur.apache.org (HELO minotaur.apache.org) (140.211.11.9) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 07 Jul 2011 10:17:29 +0000 Received: from localhost (HELO mail-iw0-f180.google.com) (127.0.0.1) (smtp-auth username rnewson, mechanism plain) by minotaur.apache.org (qpsmtpd/0.29) with ESMTP; Thu, 07 Jul 2011 10:17:28 +0000 Received: by iwn9 with SMTP id 9so1396838iwn.11 for ; Thu, 07 Jul 2011 03:17:28 -0700 (PDT) MIME-Version: 1.0 Received: by 10.231.74.18 with SMTP id s18mr576221ibj.135.1310033848271; Thu, 07 Jul 2011 03:17:28 -0700 (PDT) Received: by 10.231.146.80 with HTTP; Thu, 7 Jul 2011 03:17:28 -0700 (PDT) In-Reply-To: References: <4E150382.2050605@gmail.com> Date: Thu, 7 Jul 2011 11:17:28 +0100 Message-ID: Subject: Re: Iterating all documents From: Robert Newson To: user@couchdb.apache.org Content-Type: text/plain; charset=ISO-8859-1 It would be better to read through _changes than _all_docs, surely? :) B. On 7 July 2011 10:03, Dan Sheedy wrote: > oh. > > Sorry for the thread high jack. Good haircut btw, no regrets. > > On Thu, Jul 7, 2011 at 5:28 PM, Max Ogden wrote: > >> you should reconsider the haircut >> >> On Thu, Jul 7, 2011 at 12:25 AM, Dan Sheedy wrote: >> >> > im heading off. might be late dinner. I'm getting my haircut. should be >> > home >> > 8ish. >> > >> > On Thu, Jul 7, 2011 at 10:53 AM, Patrick Barnes >> wrote: >> > >> > > Alternatively, how about you page through _all_docs? >> > > >> > > 1. Query http://server:5984/dbname/_**all_docs?limit=100< >> > http://server:5984/dbname/_all_docs?limit=100> >> > > >> > > 2. Process that set. Store the id of the last document. >> > > >> > > 3. Query http://server:5984/dbname/_**all_docs?limit=100&startkey=(** >> > > last_id)&skip=1< >> > http://server:5984/dbname/_all_docs?limit=100&startkey=(last_id)&skip=1 >> >to >> > get the next set. >> > > >> > > 4. Repeat 2 and 3 until the returned set is empty. >> > > >> > > If your batch processing has to be able to resume after being >> terminated, >> > > just store the last_id in a file between each set. >> > > >> > > If you documents come from a specific view, you can do that too, the >> only >> > > difference would be that 'startkey' needs to be the last record's view >> > key, >> > > and you may also need a 'startkey_docid=(last_id)' parameter if the >> keys >> > are >> > > not unique. >> > > >> > > -Patrick >> > > >> > > >> > > On 7/07/2011 9:41 AM, Matthias Eck wrote: >> > > >> > >> Hello, >> > >> >> > >> I need to add a new field to all documents in my database. >> > >> >> > >> To have a better control I wanted to do this by batches and defined 2 >> > >> views: >> > >> documents_without_newfield >> > >> documents_with_newfield >> > >> >> > >> My idea was to just take the first 100 returned by the >> > >> documents_without_newfield view, calculate the new field for all of >> > >> them, save them and take the next 100 etc. >> > >> >> > >> As it turns out the views do not seem to be updated immediately, which >> > >> means that the view documents_without_newfield returns about 95 >> > >> documents that actually already had the new field calculated in the >> > >> previous step. >> > >> >> > >> Can I force the view to update immediately so I can iterate through >> > >> the all documents? >> > >> >> > >> Thanks, >> > >> Matthias >> > >> >> > >> >> > >> >