From user-return-5452-apmail-couchdb-user-archive=couchdb.apache.org@couchdb.apache.org Thu Jul 09 08:14:02 2009 Return-Path: Delivered-To: apmail-couchdb-user-archive@www.apache.org Received: (qmail 96828 invoked from network); 9 Jul 2009 08:14:02 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 9 Jul 2009 08:14:02 -0000 Received: (qmail 53640 invoked by uid 500); 9 Jul 2009 08:14:11 -0000 Delivered-To: apmail-couchdb-user-archive@couchdb.apache.org Received: (qmail 53558 invoked by uid 500); 9 Jul 2009 08:14:11 -0000 Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@couchdb.apache.org Delivered-To: mailing list user@couchdb.apache.org Received: (qmail 53548 invoked by uid 99); 9 Jul 2009 08:14:11 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 09 Jul 2009 08:14:11 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of b.candler@pobox.com designates 208.72.237.25 as permitted sender) Received: from [208.72.237.25] (HELO sasl.smtp.pobox.com) (208.72.237.25) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 09 Jul 2009 08:14:00 +0000 Received: from localhost.localdomain (unknown [127.0.0.1]) by a-sasl-quonix.sasl.smtp.pobox.com (Postfix) with ESMTP id 7E1B726192; Thu, 9 Jul 2009 04:13:39 -0400 (EDT) Received: from mappit (unknown [80.45.95.114]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by a-sasl-quonix.sasl.smtp.pobox.com (Postfix) with ESMTPSA id 20CF726190; Thu, 9 Jul 2009 04:13:38 -0400 (EDT) Received: from brian by mappit with local (Exim 4.69) (envelope-from ) id 1MOol6-0004Ao-Jm; Thu, 09 Jul 2009 09:13:36 +0100 Date: Thu, 9 Jul 2009 09:13:36 +0100 From: Brian Candler To: Daniel =?iso-8859-1?Q?Tr=FCmper?= Cc: user@couchdb.apache.org Subject: Re: Updating views while inserting documents Message-ID: <20090709081336.GB15577@uk.tiscali.com> References: <6D108219-1BAE-43EF-BEE2-6D0DE33257DB@googlemail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <6D108219-1BAE-43EF-BEE2-6D0DE33257DB@googlemail.com> User-Agent: Mutt/1.5.17+20080114 (2008-01-14) X-Pobox-Relay-ID: 68966224-6C60-11DE-B261-DC021A496417-28021239!a-sasl-quonix.pobox.com X-Virus-Checked: Checked by ClamAV on apache.org >> What version of couchdb? What version of erlang? What version of >> spidermonkey? What O/S? > CouchDB: 0.9.1 (from the svn branch) > Erlang: 5.6.5 OTP: R12B5 > Spidermokey: 1.8.1.16 > OS: Uuntu Jaunty (9.04) Looks fine except there may or may not be memory usage problems with spidermonkey 1.8. I am using 1.7 (the default in Jaunty) Is it a 64-bit OS? I think I read here a suggestion to compile Erlang in 32-bit mode, but you'll have to check the archives. I still don't own a 64-bit machine :-) >> Try updating to couchdb trunk at least erlang 12b5. > So the 0.9.x branch is not recommended? I think it should be fine, but it would be a useful data point to compare to head. > A few more details. The database I am building the index for has > approximately 127.000 documents (240 MB). Until about 40% the index is > built without problems. After that it slows down really fast. Are you swapping? Try 'top' then press capital M to look for memory hogs, and 'vmstat 2' to watch I/O. > The views itself are quite simple. I currently have several views and > the emit functions are quite simple: > > emit( field, null ); > > emit( doc._id, null ); > > emit( [field, doc[field]], null ); > > emit( field, doc[field] ); Anything in the logs when it starts to slow down? Aside: I would write emit( [field, doc[field] || null], null ); in case doc doesn't have a [field] member. The view server barfs its guts if you try to get it to serialise 'undefined'. Other than that, no I'm afraid I don't have any ideas. Regards, Brian.