Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
Received-SPF: pass (athena.apache.org: domain of paul.joseph.davis@gmail.com
 designates 74.125.44.28 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type:content-transfer-encoding;
        b=IvwHj8wPF68Km6qU3QzjqRTfnpSjkxdB/PlRCFk5aLL5rpcJ7n0y8Yss9NacIMTNCN
         DEWYx0MgML26tmTWSpoBPKpu3bXovqCwk28ygFMbcYOaEUTZv9K38q4iyAxGXmpmnDQw
         mK/fyFLYWyEvpYt63PzQEHmUHHHFuB80uTGic=
MIME-Version: 1.0
In-Reply-To: <DE17C40B-C48B-4842-9579-2A10B3EE76AA@apache.org>
References: <e2111bbb0905101849y546cd0c6y7af79e57f3faf8fb@mail.gmail.com>
	 <D824EF58-5B07-4154-8A63-8B6DFD0D85CF@apache.org>
	 <DC845851-E066-40E3-ACA4-467DCED1C9A3@gmail.com>
	 <e282921e0905111352v53e5680du3c474612f8b7eb8d@mail.gmail.com>
	 <DE17C40B-C48B-4842-9579-2A10B3EE76AA@apache.org>
Date: Wed, 13 May 2009 16:01:31 -0400
Message-ID: <e2111bbb0905131301t7da2d791u44fba272ed3049b4@mail.gmail.com>
Subject: Re: chunkify profiling (was Re: Patch to couch_btree:chunkify)
From: Paul Davis <paul.joseph.davis@gmail.com>
To: dev@couchdb.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Adam,

No worries about the delay. I'd agree that the first graph doesn't
really show much than *maybe* we can say the patch reduces the
variability a bit.

On the second graph, I haven't the faintest why that'd be as such.
I'll have to try and setup fprof and see if I can figure out what
exactly is taking most of the time. Perhaps we're looking at wrong
thing by reducing term_to_binary. You did say that most of the time
was spent in size/1 as opposed to term_to_binary the other day which
is hard to believe at best.

I'll put this on the weekend agenda. Until I can show that its
consistently faster I'll hold off.

For reference, when you say 2K docs in batches of 1K, did you mean 200K?

Also, note to self, we should check speeds for dealing with uuids too
to see if the non-ordered mode makes a difference.

Paul

On Wed, May 13, 2009 at 3:33 PM, Adam Kocoloski <kocolosk@apache.org> wrote=
:
> Sorry for the delay on this front. =A0I ran hovercraft:lightning 20 times=
 each
> with and without Paul's patch. =A0Each run inserted 2k docs in batches of
> 1000. =A0Here are two plots showing the effect of the patch:
>
> http://dl.getdropbox.com/u/237885/insert_rate.png
> http://dl.getdropbox.com/u/237885/chunkify_fraction.png
>
> The first plot histograms the insert rate for the two scenarios*. =A0I do=
n't
> really see much of a difference. =A0The second plot uses fprof to plot th=
e
> fraction of time the couch_db_updater process spent in chunkify and any
> functions called by chunkify. =A0For those familiar with fprof, it's the =
ratio
> of ACC for couch_btree:chunkify/2 divided by OWN for the updater pid. =A0=
If
> fprof is to be believed, the trunk code is almost 2x faster.
>
> Adam
>
> * the reason the insert rate is so low is because fprof is running. =A0Tu=
rning
> off the profiler speeds things up by an order of magnitude, in accord wit=
h
> the numbers Chris has posted.
>