Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
Received-SPF: pass (athena.apache.org: domain of fdmanana@gmail.com designates
 209.85.161.54 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:sender:in-reply-to:references:date
         :x-google-sender-auth:message-id:subject:from:to:content-type
         :content-transfer-encoding;
        b=vFKWkUhdoBW2c6hPQGGJqXNvXsw7YI+nFz3jaFgnPWAN0y7q7pFJ0wHiJE52JmEHDE
         XfS4uhGOztFyQM78DcfvyHxdqboZQsMwqcAdQcnPILBZ5yov9sfxtu+WAX9vcLD+JDJU
         gEL65QB+oWz6T6C6fczYLQ1VArX4pvt2fypIY=
MIME-Version: 1.0
Sender: fdmanana@gmail.com
In-Reply-To: <0E843BFE-083F-4F33-ADBC-1480E921AF5C@apache.org>
References: <AANLkTim5uNVkSCVhHuu_1GA1xKVFJFBWE2-Wtb4vVx8g@mail.gmail.com>
	<F51B279B-5306-405E-8355-25DBCA8A1F0A@apache.org>
	<0E843BFE-083F-4F33-ADBC-1480E921AF5C@apache.org>
Date: Mon, 13 Dec 2010 14:05:26 +0000
Message-ID: <AANLkTinZ_t-+O2Lp_REL_OS1kV0BwOA7k1y1WrV3tAnF@mail.gmail.com>
Subject: Re: worst performance in 1.1.x (compared to 1.0.x)
From: Filipe David Manana <fdmanana@apache.org>
To: dev@couchdb.apache.org
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Adam,

My tests were done with OTP R14B, dual-core Thinkpad (with Linux) and
a 5400 rpms hard disk.

I think the cause for such huge differences between our tests is the
OS. The time I spent with Joel, we always had very different results
for the same tests. He also used a Macbook pro (with OS X). (I've
heard several times that OS X's IO scheduler is worst than Linux's
one).

It's weird that in your 2nd graph, trunk reads are a lot worse
compared 4b0948d-after-R14B01-reads.
Do you also have a Linux machine to test this?


On Mon, Dec 13, 2010 at 3:02 AM, Adam Kocoloski <kocolosk@apache.org> wrote=
:
> I tried R14B01, the custom small_doc.json, and lowering schedulers_online=
 to 1, but the writers continue to starve. =C2=A0Trunk (4137a8e) does not s=
tarve the writers, almost certainly due to your updater_fd patch. =C2=A0Com=
paring trunk and the mochiweb upgrade commit I get
>
> http://graphs.mikeal.couchone.com/#/graph/df0f79455c9c600f66d1ce42ea018d7=
e
>
> I think it's still important that you observe a performance regression wi=
th the introduction of that patch, but =C2=A0something else in our respecti=
ve setups clearly has a much greater effect on the results. =C2=A0Regards,
>
> Adam
>
> On Dec 12, 2010, at 8:42 PM, Adam Kocoloski wrote:
>
>> Hi Filipe, I cannot reproduce those results at all, though I didn't try =
loading the custom small_doc.json. =C2=A0If I use 200 readers and 100 write=
rs the writers are completely starved and I get e.g.
>>
>> http://graphs.mikeal.couchone.com/#/graph/df0f79455c9c600f66d1ce42ea016e=
07
>>
>> I need to lower the readers down to ~20 to keep the write throughput rea=
sonable. =C2=A0I'm running R13B04 on a dual-core OS X 10.6 MacBook. =C2=A0I=
'll try a few more things including the custom doc and R14B01, but do you h=
ave suggestions for why these results might be so dramatically different?
>>
>> Adam
>>
>> On Dec 12, 2010, at 7:17 AM, Filipe David Manana wrote:
>>
>>> Hi,
>>>
>>> While running a relaximation test to compare read/write performance
>>> between 1.1.x and 1.0.x, I found out that 1.1.x has worst performance
>>>
>>> It seems the cause is related to the new Mochiweb version introduced
>>> in commit 4b0948ddb3a428f8a5330e05745b2fbd4ccf9375 -
>>> https://github.com/apache/couchdb/commit/4b0948ddb3a428f8a5330e05745b2f=
bd4ccf9375
>>>
>>> Comparing the performance of this revision with the previous one
>>> (cd214b23e8129868d4a7020ddafd55a16e496652), I get the following
>>> results:
>>>
>>> http://graphs.mikeal.couchone.com/#/graph/df0f79455c9c600f66d1ce42ea012=
5e5
>>>
>>> Both read and write performance are worse for
>>> 4b0948ddb3a428f8a5330e05745b2fbd4ccf9375.
>>>
>>> The cause could be the configuration we pass to Mochiweb in
>>> couch_httpd. The new Mochiweb sets the nodelay option to false by
>>> default and it uses now several acceptor processes (16 by default).
>>> However even with the following small patch I still get about the same
>>> disappointing results:
>>>
>>> diff --git a/src/couchdb/couch_httpd.erl b/src/couchdb/couch_httpd.erl
>>> index 23ff7f9..e93c7e7 100644
>>> --- a/src/couchdb/couch_httpd.erl
>>> +++ b/src/couchdb/couch_httpd.erl
>>> @@ -97,7 +97,9 @@ start_link(Name, Options) ->
>>> =C2=A0 =C2=A0{ok, Pid} =3D case mochiweb_http:start(Options ++ [
>>> =C2=A0 =C2=A0 =C2=A0 =C2=A0{loop, Loop},
>>> =C2=A0 =C2=A0 =C2=A0 =C2=A0{name, Name},
>>> - =C2=A0 =C2=A0 =C2=A0 =C2=A0{ip, BindAddress}
>>> + =C2=A0 =C2=A0 =C2=A0 =C2=A0{ip, BindAddress},
>>> + =C2=A0 =C2=A0 =C2=A0 =C2=A0{nodelay, true},
>>> + =C2=A0 =C2=A0 =C2=A0 =C2=A0{acceptor_pool_size, 32}
>>> =C2=A0 =C2=A0]) of
>>> =C2=A0 =C2=A0{ok, MochiPid} -> {ok, MochiPid};
>>> =C2=A0 =C2=A0{error, Reason} ->
>>>
>>>
>>> (Also tried higher values for acceptor_pool_size, which doesn't help)
>>>
>>> The test was run like this:
>>>
>>> $ node tests/compare_write_and_read.js --wclients 100 --rclients 200 \
>>> -name1 new_4b0948ddb3a428f8a5330e05745b2fbd4ccf9375 -name2
>>> old_cd214b23e8129868d4a7020ddafd55a16e496652 \
>>> -url1 http://localhost:5984/ -url2 http://localhost:5985/ \
>>> --duration 120
>>>
>>> With the following document template (file
>>> relaximation/common/small_doc.json):
>>> http://friendpaste.com/7GKUEg0SZHmOf0g7Dh5IWC
>>>
>>> Can anyone confirm these results?
>>> If confirmed, I would say this is a blocker for 1.1.0.
>>>
>>> thanks
>>>
>>>
>>> --
>>> Filipe David Manana,
>>> fdmanana@gmail.com, fdmanana@apache.org
>>>
>>> "Reasonable men adapt themselves to the world.
>>> Unreasonable men adapt the world to themselves.
>>> That's why all progress depends on unreasonable men."
>>
>
>


--=20
Filipe David Manana,
fdmanana@gmail.com, fdmanana@apache.org

"Reasonable men adapt themselves to the world.
=C2=A0Unreasonable men adapt the world to themselves.
=C2=A0That's why all progress depends on unreasonable men."