Return-Path: Delivered-To: apmail-couchdb-user-archive@www.apache.org Received: (qmail 6059 invoked from network); 9 Jun 2009 09:25:12 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 9 Jun 2009 09:25:12 -0000 Received: (qmail 94398 invoked by uid 500); 9 Jun 2009 09:25:23 -0000 Delivered-To: apmail-couchdb-user-archive@couchdb.apache.org Received: (qmail 94303 invoked by uid 500); 9 Jun 2009 09:25:22 -0000 Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@couchdb.apache.org Delivered-To: mailing list user@couchdb.apache.org Received: (qmail 94293 invoked by uid 99); 9 Jun 2009 09:25:22 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 09 Jun 2009 09:25:22 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of b.candler@pobox.com designates 207.106.133.19 as permitted sender) Received: from [207.106.133.19] (HELO sasl.smtp.pobox.com) (207.106.133.19) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 09 Jun 2009 09:25:13 +0000 Received: from localhost.localdomain (unknown [127.0.0.1]) by a-sasl-fastnet.sasl.smtp.pobox.com (Postfix) with ESMTP id ECB66BAB2E; Tue, 9 Jun 2009 05:24:50 -0400 (EDT) Received: from mappit (unknown [79.69.85.237]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by a-sasl-fastnet.sasl.smtp.pobox.com (Postfix) with ESMTPSA id 50191BAB2D; Tue, 9 Jun 2009 05:24:49 -0400 (EDT) Received: from brian by mappit with local (Exim 4.69) (envelope-from ) id 1MDxZX-0004Rj-GE; Tue, 09 Jun 2009 10:24:47 +0100 Date: Tue, 9 Jun 2009 10:24:47 +0100 From: Brian Candler To: Jan Lehnardt Cc: user@couchdb.apache.org Subject: Re: Optimizing view building Message-ID: <20090609092447.GA14982@uk.tiscali.com> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.17+20080114 (2008-01-14) X-Pobox-Relay-ID: 622ACEDE-54D7-11DE-8848-97731A10BFE7-28021239!a-sasl-fastnet.pobox.com X-Virus-Checked: Checked by ClamAV on apache.org On Sat, Jun 06, 2009 at 12:18:34AM +0200, Jan Lehnardt wrote: > beam & couchjs wait for each other while the respective partner process > is working. There is room for optimization, we haven't gotten around to > that yet :) > > See this and the dev@ mailing list archives for detailed discussions. I would very much like to see this. I have a small test database with only ~1,400 documents, expanding to maybe ~12,000 key/value pairs. On my 1.2GHz laptop it takes only about 2 seconds to bulk-load these documents, but over 90 seconds to index them. My first job will be to turn this into a proper reproducible benchmark, and then to see how much erlview improves it. But I'm hoping that some sort of batching of view processing would make as big a difference to view performance as it does when bulk loading. (Perhaps send 1MB of documents at a time to the view server?) Regards, Brian.