From couchdb-user-return-300-apmail-incubator-couchdb-user-archive=incubator.apache.org@incubator.apache.org Mon Apr 28 06:50:32 2008 Return-Path: Delivered-To: apmail-incubator-couchdb-user-archive@locus.apache.org Received: (qmail 44349 invoked from network); 28 Apr 2008 06:50:32 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 28 Apr 2008 06:50:32 -0000 Received: (qmail 61137 invoked by uid 500); 28 Apr 2008 06:50:33 -0000 Delivered-To: apmail-incubator-couchdb-user-archive@incubator.apache.org Received: (qmail 61108 invoked by uid 500); 28 Apr 2008 06:50:33 -0000 Mailing-List: contact couchdb-user-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: couchdb-user@incubator.apache.org Delivered-To: mailing list couchdb-user@incubator.apache.org Received: (qmail 61097 invoked by uid 99); 28 Apr 2008 06:50:33 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 27 Apr 2008 23:50:33 -0700 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [208.97.132.202] (HELO randymail-a12.g.dreamhost.com) (208.97.132.202) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 28 Apr 2008 06:49:38 +0000 Received: from [10.85.161.42] (unknown [32.156.40.25]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by randymail-a12.g.dreamhost.com (Postfix) with ESMTP id A4D6AA6EF4; Sun, 27 Apr 2008 23:49:56 -0700 (PDT) From: Cortland Klein To: "couchdb-user@incubator.apache.org" In-Reply-To: X-Mailer: iPhone Mail (4A102) Subject: Re: Updating views on save Content-Transfer-Encoding: 7bit References: Message-Id: <9DD76E5A-88B7-4970-B0D0-449966548DA6@pixelcort.com> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Mime-Version: 1.0 (iPhone Mail 4A102) Date: Sun, 27 Apr 2008 23:49:32 -0700 Cc: "couchdb-user@incubator.apache.org" X-Virus-Checked: Checked by ClamAV on apache.org Not sure why your views are timing out, but from my current understanding views are incrementally updated with modifications but only incrementally updated on a call to that view. One hypothesis is that after the thousands of inserts the next view call is taking a long time doing the incremental updates to the view since the last time it was called. Are you using the javascript spodermonkey viewserver(default) or another one? Check the complexity of the view and possibly minimize the view's complexity. As a final thought iirc when querying a view in a design document, all views in that designdocument are called. Is it possible you might have unrelated views in the same design document? I'm not sure, but I think the pattern here is you put views most likely to be called near to each other in the design document, say blog summaries view followed by full content view, and have less related views in a different design document, say for a list of authors or a tag list. Hope this helps. Cortland Klein +1 408 506 9791 http://pixelcort.com/ Sent from my iPhone On Apr 27, 2008, at 4:04 PM, Guby wrote: > Hello dear Couchers > > I understand that the views are indexed the first time they are > accessed and as far as I know there is no way to turn on view > updating on document save. I really don't understand the reasoning > behind this behavior. The advantage of the pre-populated/indexed > views are that they are blazingly fast to query and access, but that > advantage disappears when the first request after a document update > has to regenerate the view first! > > I am currently building a web app where the background processes > perform a lot of writes to the database. The time it takes to write > a document is not critical for me. What is critical though is the > time it takes to load web pages for the end user that require > content from the database. > In some situations the background processes add thousands of > documents to the database within a short period of time, and when > the user tries to access a page after such an update the view > querying sometimes takes minutes and as a consequence of that the > browser times out... Not a recipe for happy customers... > > The only solution I can see at the moment is to create a worker that > queries the database whenever it is told that there has been a > document update, but that seems really stupid and unnecessary. And > in my case, running on a smallish VPS, a big waste of resources > having an extra working doing something the database itself could > just as well have done. It also requires a lot of extra coding > notifying the worker whenever I update or create a document > throughout my app. > > I am sure you have reasons for having implemented the views the way > you have, but I would be really interested to hear why it has been > done this way! > > My wishes are for an optional updating of views on save feature! In > some cases that might regenerate a view several times without it > actually being accessed in between, but that is a tradeoff I can > live with, slow views on the other hand is something I can not! > > All the best > Sebastian