From user-return-13021-apmail-couchdb-user-archive=couchdb.apache.org@couchdb.apache.org Fri Oct 01 12:30:08 2010 Return-Path: Delivered-To: apmail-couchdb-user-archive@www.apache.org Received: (qmail 30594 invoked from network); 1 Oct 2010 12:30:08 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 1 Oct 2010 12:30:08 -0000 Received: (qmail 95492 invoked by uid 500); 1 Oct 2010 12:30:07 -0000 Delivered-To: apmail-couchdb-user-archive@couchdb.apache.org Received: (qmail 95224 invoked by uid 500); 1 Oct 2010 12:30:04 -0000 Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@couchdb.apache.org Delivered-To: mailing list user@couchdb.apache.org Received: (qmail 95216 invoked by uid 99); 1 Oct 2010 12:30:03 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Oct 2010 12:30:03 +0000 X-ASF-Spam-Status: No, hits=0.3 required=10.0 tests=SPF_PASS,URIBL_RHS_DOB X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of paul.hirst@sophos.com designates 213.31.172.35 as permitted sender) Received: from [213.31.172.35] (HELO mx5.sophos.com) (213.31.172.35) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Oct 2010 12:29:56 +0000 Received: from mx5.sophos.com (localhost.localdomain [127.0.0.1]) by localhost (Postfix) with SMTP id 4AFD3685B7; Fri, 1 Oct 2010 13:29:22 +0100 (BST) Received: from uk-exch2.green.sophos (uk-exch2.green.sophos [10.100.199.17]) by mx5.sophos.com (Postfix) with ESMTP id 274C568565; Fri, 1 Oct 2010 13:29:22 +0100 (BST) Received: from [10.181.0.129] (10.181.0.129) by uk-exch2.green.sophos (10.100.199.10) with Microsoft SMTP Server (TLS) id 8.1.340.0; Fri, 1 Oct 2010 13:29:22 +0100 Subject: Re: Spontaneous reindex and View Group Indexer Finishing taking a long time From: Paul Hirst To: "user@couchdb.apache.org" CC: "adrian@adrianpemsel.de" In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Date: Fri, 1 Oct 2010 13:29:21 +0100 Message-ID: <1285936161.28367.140.camel@meerkat.green.sophos> MIME-Version: 1.0 X-Mailer: Evolution 2.28.3 Content-Transfer-Encoding: quoted-printable DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sophos.com; h=subject:from:to:cc:in-reply-to:references:content-type:date:message-id:mime-version:content-transfer-encoding; s=global; bh=HfemxEgZqKtvfe9KZmHA5fwASGdKq5OVRasb53Kz+aI=; b=Mgwf1d8FpBLtBtyI/Rb1sKJht6EuDzMxJeyRbH1g7oVjZa9qurhnJtHUaFozbIvS0djmhkiXNi/ze8a/+7yclQPtY4eeK4i5uDDAWARZeqOij+B8xM+svFEUPq1jm7gNLjnSINoJVrInkhRDEhPbeX1pzSiRx46IBeMN14AEozo= X-Virus-Checked: Checked by ClamAV on apache.org On Fri, 2010-10-01 at 12:19 +0100, David Coallier wrote: > For instance, consider this document: > { > _id: "xxx", > "name": "example1", > "age": 23 > } > > and > > { > _id: "yyy", > "name": "example2" > } > > Then the following map and reduce functions: > map =3D function(doc) { > emit(doc.name, doc.age); > } > > This previous case is going to generate an exception in the javascript > engine, stopping the indexing and re-spawning another process picking > up the indexing where it was. This caused some massive interruptions > in our indexing. Is this true? I had naively assumed that this behaviour was a 'feature' I could use. Ie I could avoid some if clauses and just rely on exceptions being thrown when, for example, when a view is only interested in certain types of document. For example documents like { _id: "person_paul", name: "paul", lives_in: [ "city_london" ], } { _id: "city_london", info: { city_name: "london", population: "lots", } } I has assumed it was reasonable to write a view which returns all cities with their populations as map =3D function(doc) { emit(doc.info.city_name, doc.info.population); } doc.info will be undefined for 'person' records and thus doc.info.city_name will throw an exception. I had assumed this would cheaply skip over 'person' records, because an exception would be thrown but it sounds like this exception is actually really expensive? Can anyone clarify? Thanks. Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon, OX14 3YP, United= Kingdom. Company Reg No 2096520. VAT Reg No GB 348 3873 20.