From user-return-11749-apmail-couchdb-user-archive=couchdb.apache.org@couchdb.apache.org Wed Jul 28 09:02:46 2010 Return-Path: Delivered-To: apmail-couchdb-user-archive@www.apache.org Received: (qmail 485 invoked from network); 28 Jul 2010 09:02:45 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 28 Jul 2010 09:02:45 -0000 Received: (qmail 57145 invoked by uid 500); 28 Jul 2010 09:02:44 -0000 Delivered-To: apmail-couchdb-user-archive@couchdb.apache.org Received: (qmail 56786 invoked by uid 500); 28 Jul 2010 09:02:41 -0000 Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@couchdb.apache.org Delivered-To: mailing list user@couchdb.apache.org Received: (qmail 56777 invoked by uid 99); 28 Jul 2010 09:02:39 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 28 Jul 2010 09:02:39 +0000 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests=FREEMAIL_FROM,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of randall.leeds@gmail.com designates 209.85.161.52 as permitted sender) Received: from [209.85.161.52] (HELO mail-fx0-f52.google.com) (209.85.161.52) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 28 Jul 2010 09:02:34 +0000 Received: by fxm8 with SMTP id 8so1480559fxm.11 for ; Wed, 28 Jul 2010 02:02:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:content-type; bh=Sx/nm61vYibe9Pk3dvYJ6RFx4gbK6W5X58U66W19emg=; b=l6ezyv3TuC9/rzdVhRnD4YMmjaa0EqWk8IpOQD/yVFfWujKynpfNlzU65W8+sM9zLi Nw4ZOxPMrlW6E2/rkjZL8E6IVc5B9bbmn43AV8ZUgcD4DNmNz2G0L1BkYuYxctA6OISe QOAb5iweLbDA1GL78vwntDYdaJ0hlOBTWif/U= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=QjWg1RbZIeir34dHRBbxcL6aEwhypcNHIKjn4S8COo7lLrYFJprGjqhzeFI2Ad4yD1 mXJghPo8ccm+jiOPvcLdC3d6g0fNCMaFqIwVhESATvfUpknBRXcQXpIqZnFbo/EmJuiI 5akvsLWorK/Bwx51xlbrlJYBaDZ1SAIObPqa4= MIME-Version: 1.0 Received: by 10.239.165.13 with SMTP id v13mr662473hbd.198.1280307732792; Wed, 28 Jul 2010 02:02:12 -0700 (PDT) Received: by 10.239.187.196 with HTTP; Wed, 28 Jul 2010 02:02:12 -0700 (PDT) In-Reply-To: References: Date: Wed, 28 Jul 2010 02:02:12 -0700 Message-ID: Subject: Re: beam CPU hog From: Randall Leeds To: user@couchdb.apache.org Content-Type: text/plain; charset=UTF-8 On Wed, Jul 28, 2010 at 01:41, Sivan Greenberg wrote: > Just another note, the problem see to grow larger when the database > size expands. I am going to time each operation now to see if I can > find a specific culprit. I looked at your code and I've got a couple things for you to try. 1) Spinning up a replication means a bunch of HTTP requests. The fact that the requests are local only means you're not seeing network latency and your cpu is more pinned than in the real world. You could try creating 100 documents first and causing conflicts on all of them in a single replication or replace your urls with bare db names (i.e., just 'session_store_rep' and 'session_store') to get local replication which bypasses the HTTP layer. The latter option will also reduce the amount of json encoding/decoding you're doing. 2) Give us more info about your problem with 'load'. You really shouldn't care about the cpu load. How long your test takes is much more important. If you're getting a decent number of operations/second and your cpu is pinned you should be thrilled. Imagine if you were encoding an audio file and your cpu wasn't at 100%. You would be mad because it would take longer than it has to. In general, if you can be cpu bound you're doing things right as long as things are humming along quickly. It means couchdb is fulfilling your request as quickly as possible and neither the network nor the disk is a bottleneck. Hoping this helps you out :) Randall