Return-Path: X-Original-To: apmail-couchdb-dev-archive@www.apache.org Delivered-To: apmail-couchdb-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id AC445C680 for ; Fri, 11 May 2012 12:35:57 +0000 (UTC) Received: (qmail 74536 invoked by uid 500); 11 May 2012 12:35:57 -0000 Delivered-To: apmail-couchdb-dev-archive@couchdb.apache.org Received: (qmail 74502 invoked by uid 500); 11 May 2012 12:35:57 -0000 Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@couchdb.apache.org Delivered-To: mailing list dev@couchdb.apache.org Received: (qmail 74493 invoked by uid 99); 11 May 2012 12:35:57 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 11 May 2012 12:35:57 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of djc.ochtman@gmail.com designates 74.125.82.180 as permitted sender) Received: from [74.125.82.180] (HELO mail-we0-f180.google.com) (74.125.82.180) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 11 May 2012 12:35:49 +0000 Received: by weyt11 with SMTP id t11so432723wey.11 for ; Fri, 11 May 2012 05:35:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:content-type; bh=QWCbL0VKb3wEVKvwd0carZPTSA9OaL1drP1yHQMaEHQ=; b=gFnPZjfD4J1ZtWBIWK5L9UsM4y93OhP0GeRSbvdXLLCwMEzQXqNbMXOgPUZScWZvfN SGJsnG9GwWVQt1Ua9sI9MHmDBiSeArF8JXmjKGrwDTj91dWXT6wYbYtE83Gw5yO33asg jFEb79ImgSFipvqKCvvxAuxG0pg4g0dpIStYjOpVPQEqtawzALdsgfvPHdLSk1eCQR2U pG//AJFKkCNzOGKwrtbrvBeOQz4DCI41xuJ7dVFD8jvNIskt/fQuxjHvj/LdCVymur4q QFnBofICYeo5IB9Mc1Qi5Fm1z5WV0Zxjj1iBra1QUZ/pFgfZRj6j2spax4vBihs0huQQ ZwQA== Received: by 10.180.102.3 with SMTP id fk3mr7391309wib.9.1336739729294; Fri, 11 May 2012 05:35:29 -0700 (PDT) MIME-Version: 1.0 Sender: djc.ochtman@gmail.com Received: by 10.216.137.162 with HTTP; Fri, 11 May 2012 05:35:09 -0700 (PDT) In-Reply-To: References: From: Dirkjan Ochtman Date: Fri, 11 May 2012 14:35:09 +0200 X-Google-Sender-Auth: PMeLeER7KiMocSmBkn1NonyPYsQ Message-ID: Subject: Re: Post-mortem To: dev@couchdb.apache.org Content-Type: text/plain; charset=UTF-8 On Fri, May 11, 2012 at 2:29 PM, Robert Newson wrote: > Making a single view indexing process faster keeps coming up. For one > thing, it's not that easy, otherwise it would have been done by now. > For another, this problem vanishes when you shard (and the BigCouch > merge will bring this to CouchDB). What does sharding mean in this context? Running CouchDB/BigCouch on multiple servers, or just running multiple processes on a single box? If the latter, why can't we run multiple threads/Erlang processes within a single shard/OS process? If the former, that's kind of silly, in the sense that building indexes (at least for me) is CPU-bound but leaves many of the cores in my server idle. Cheers, Dirkjan