From dev-return-4930-apmail-couchdb-dev-archive=couchdb.apache.org@couchdb.apache.org Sun Jul 05 08:59:20 2009 Return-Path: Delivered-To: apmail-couchdb-dev-archive@www.apache.org Received: (qmail 13144 invoked from network); 5 Jul 2009 08:59:20 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 5 Jul 2009 08:59:20 -0000 Received: (qmail 81382 invoked by uid 500); 5 Jul 2009 08:59:30 -0000 Delivered-To: apmail-couchdb-dev-archive@couchdb.apache.org Received: (qmail 81313 invoked by uid 500); 5 Jul 2009 08:59:30 -0000 Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@couchdb.apache.org Delivered-To: mailing list dev@couchdb.apache.org Received: (qmail 81303 invoked by uid 99); 5 Jul 2009 08:59:30 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 05 Jul 2009 08:59:30 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of paul.joseph.davis@gmail.com designates 209.85.210.204 as permitted sender) Received: from [209.85.210.204] (HELO mail-yx0-f204.google.com) (209.85.210.204) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 05 Jul 2009 08:59:22 +0000 Received: by yxe42 with SMTP id 42so1509326yxe.13 for ; Sun, 05 Jul 2009 01:59:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=wq/2KzxkQr/7mEfthez8FKDuj1j63xpbcE4pvth+YKU=; b=Mo7rF8Ysxde4ofbBXdPCctbzmLeUpo3wOsAEkQGNMwnKFZlbXXvDubHzzZjFs+mElR jx5HMvNH6t3Bvm3u52ugx+l8264Z1jQLlrQFxYvRTHAP9c+alqnDwbPj8pRGNqlX9i1a yi+IVTX9cASeiO59eEVRweAHe09qk3npaX9LE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=NY/0SDtLdZIFcNqZoVvpBCp1Bw84C8bXoL+2NuvgxZFQphRxwfLkyW29WeGQVMs2TX UdjF9f2pDx0INbxnYUJ5CFh/anO9lXLi7F1sQZQvRqOC/mOfUZ2uWZk8MOLQwLTWJVIM 73UtFc7f9aT5WJn3Tx0efhtEY+5bKgxml5dp0= MIME-Version: 1.0 Received: by 10.100.42.4 with SMTP id p4mr6045005anp.115.1246784012470; Sun, 05 Jul 2009 01:53:32 -0700 (PDT) In-Reply-To: <4A505EB1.3050001@krampe.se> References: <4A4E7F53.7010406@krampe.se> <4A4FF0E4.4040804@krampe.se> <4A505EB1.3050001@krampe.se> Date: Sun, 5 Jul 2009 04:53:32 -0400 Message-ID: Subject: Re: Possible bug in indexer... (really) From: Paul Davis To: dev@couchdb.apache.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org 2009/7/5 G=F6ran Krampe : > Hi Paul! > > Paul Davis wrote: >> >> 2009/7/4 G=F6ran Krampe : >>> >>> Adam Kocoloski wrote: >>>> >>>> Not sure if it's described, but it is by design. =A0The reduce functio= n >>>> executes when the btree is modified. =A0We can't afford to cache KVs f= rom >>>> an >>>> index update in memory regardless of size; we have to set some thresho= ld >>>> when we flush them to disk. >>> >>> And I presume you can't write KVs *without* doing the reduce? >>> >>> When I wrote "described" I am referring to the blog post by Ricky Ho bt= w. >>> It >>> seems to imply a strict ordering, map -> reduce -> rereduce. IIRC. >> >> That was probably just the theoretical aspect. Map's always happen >> first obviously, and then when the key/values are inserted into the >> btree during a flush the entire tree is built which means that > 0 >> reduces are called and then re-reduces are run to fill out the tree. >> At the moment we aren't delaying re-reduce calls because it'd require >> a major overhaul to the btree code. > > Right, so there is nothing preventing delaying the reduce and rereduce ca= lls > untl the end - except of course the problem of actually implementing it. > Just want to understand. > Just to be clear, its not an implementation I expect to actually materialize. It'd require a hefty amount of reworking of the btree updates so much that it'd change some of the fundamental behavior and unless I'm just not as creative as I usually am at 5am, I can't think of a decent way of implementing such an update very effectively. > regards, G=F6ran > >