Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (athena.apache.org: domain of sivang@gmail.com designates
 209.85.160.180 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:sender:in-reply-to:references:date
         :x-google-sender-auth:message-id:subject:from:to:content-type;
        b=M30rJMwHd2fSk52Vg/+5kfUEtMhMl3Eknx7CFG8pJoyn8iZsEvM9HLLuuoHdbDYIHj
         opEpNR1W99caUlram5VkpelQmz4oN2LFajP+NK9GabelLpnhVNKRJ81q5UHXGqDy01rP
         8PuaMsa/OstkUG+AhqXTN2pzF/ECqSBijdWx8=
MIME-Version: 1.0
Sender: sivang@gmail.com
In-Reply-To: <AANLkTinmtnATahfkQ0hJmdBZt=aVt1OGqsbTKfOjMzs4@mail.gmail.com>
References: <AANLkTikBVdPSOnADN+W7v==Mdpw5FQiKBv72fnTE=zhD@mail.gmail.com>
	<AANLkTinn0-+LnD8+pHtdM1LDXaeZ4_hBMR7g2o0pF4Lc@mail.gmail.com>
	<AANLkTinmtnATahfkQ0hJmdBZt=aVt1OGqsbTKfOjMzs4@mail.gmail.com>
Date: Wed, 28 Jul 2010 12:26:24 +0300
Message-ID: <AANLkTimAmraKGQsNd-jesv3J=G6zoF+p8i-WONcZ3KU8@mail.gmail.com>
Subject: Re: beam CPU hog
From: Sivan Greenberg <sivan@omniqueue.com>
To: user@couchdb.apache.org
Content-Type: text/plain; charset=ISO-8859-1

On Wed, Jul 28, 2010 at 12:02 PM, Randall Leeds <randall.leeds@gmail.com> wrote:
> 1) Spinning up a replication means a bunch of HTTP requests. The fact
> that the requests are local only means you're not seeing network
> latency and your cpu is more pinned than in the real world. You could
> try creating 100 documents first and causing conflicts on all of them
> in a single replication or replace your urls with bare db names (i.e.,
> just 'session_store_rep' and 'session_store') to get local replication
> which bypasses the HTTP layer. The latter option will also reduce the
> amount of json encoding/decoding you're doing.

I'm actually in wish to experience closest to real world things, so
bypassing it feels to me as running the test less rigorously. Please
correct me if I'm wrong as there's a chance I didn't get to the bottom
of this still. I did change the target to not use http as this is how
it will be in the real deployment, thanks for noticing that :)

>
> 2) Give us more info about your problem with 'load'. You really
> shouldn't care about the cpu load. How long your test takes is much
> more important. If you're getting a decent number of operations/second
> and your cpu is pinned you should be thrilled.

The problem is that the servers designated to host CouchDB are very
strong servers (8Gigs of RAM, losts of CPU and cores) , but still do a
couple of others things like http server and possibly a couple more
services. So when CPU is hogged , performance of web apps is effected.

However- I would not care too much about this to start with if
CouchDB's performance would actually provide the result- the idea
about this is to have the conflict resolved in real or near real time
manner, to be as coherent as possible with the winning doc or latest
version of a shopping cart. So right now, when the db is a bit big
(less then 1G still) operations take long such that the expected
outcome of conflicts cleared (e.g. _conflicts goes away form the doc
obj when asked for with conflict=true) does not happen, when
simulating a user action triggering fetching his session details, he
gets the wrong version...

For the record, no I can't use sticky sessions :) (this has gone up
once or twice)

> Hoping this helps you out :)

This is a start :-)

Many thanks so far!

Sivan