Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
Received-SPF: pass (nike.apache.org: domain of fdmanana@gmail.com designates
 209.85.160.52 as permitted sender)
Received-SPF: pass (google.com: domain of fdmanana@gmail.com designates
 10.68.229.67 as permitted sender) client-ip=10.68.229.67;
MIME-Version: 1.0
Sender: fdmanana@gmail.com
In-Reply-To: 
 <CAN-3CBKk7rykwrwdWwizwPGOUk41xP-uYFG-i49=nZsOMZe-VQ@mail.gmail.com>
References: 
 <CAN-3CBKk7rykwrwdWwizwPGOUk41xP-uYFG-i49=nZsOMZe-VQ@mail.gmail.com>
Date: Tue, 28 Feb 2012 09:53:01 -0800
Message-ID: 
 <CAL3q7H6J+D7S751rP9Qv77yNghPUyd99rVYnNKiBgyyQYZMtqg@mail.gmail.com>
Subject: Re: Please report your indexing speed
From: Filipe David Manana <fdmanana@apache.org>
To: dev@couchdb.apache.org
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Jason, repeated my last test with the 1Kb docs (
https://gist.github.com/1930804, map function
http://friendpaste.com/5C99aqXocN6N6H1BAYIigs ) to cover branch 1.1.x
as well. Here are the full results (also in
https://gist.github.com/1930807):


Before COUCHDB-1186

fdmanana 23:21:05 ~/git/hub/slow_couchdb (master)> docs=3D500000
batch=3D5000 ./bench.sh wow.tpl
Server: CouchDB/1.2.0a-a68a792-git (Erlang OTP/R14B03)
{"couchdb":"Welcome","version":"1.2.0a-a68a792-git"}

[INFO] Created DB named `db1'
[INFO] Uploaded 5000 documents via _bulk_docs
(....)
[INFO] Uploaded 5000 documents via _bulk_docs
Building view.
{"total_rows":500000,"offset":0,"rows":[
{"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"va=
lue":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,=
"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
]}

real	5m6.676s
user	0m0.009s
sys	0m0.010s


After COUCHDB-1186

fdmanana 23:50:07 ~/git/hub/slow_couchdb (master)> docs=3D500000
batch=3D5000 ./bench.sh wow.tpl
Server: CouchDB/1.2.0a-f023052-git (Erlang OTP/R14B03)
{"couchdb":"Welcome","version":"1.2.0a-f023052-git"}

[INFO] Created DB named `db1'
[INFO] Uploaded 5000 documents via _bulk_docs
(....)
[INFO] Uploaded 5000 documents via _bulk_docs
Building view.
{"total_rows":500000,"offset":0,"rows":[
{"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"va=
lue":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,=
"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
]}

real	5m1.395s
user	0m0.008s
sys	0m0.010s


After COUCHDB-1186 + better queueing patch
(http://friendpaste.com/178nPFgfyyeGf2vtNRpL0w)

fdmanana 00:14:25 ~/git/hub/slow_couchdb (master)> docs=3D500000
batch=3D5000 ./bench.sh wow.tpl
Server: CouchDB/1.2.0a-f023052-git (Erlang OTP/R14B03)
{"couchdb":"Welcome","version":"1.2.0a-f023052-git"}

[INFO] Created DB named `db1'
[INFO] Uploaded 5000 documents via _bulk_docs
(....)
[INFO] Uploaded 5000 documents via _bulk_docs
Building view.
{"total_rows":500000,"offset":0,"rows":[
{"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"va=
lue":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,=
"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
]}

real	4m48.175s
user	0m0.008s
sys	0m0.009s


CouchDB branch 1.1.x

fdmanana 08:16:58 ~/git/hub/slow_couchdb (master)> docs=3D500000
batch=3D5000 ./bench.sh wow.tpl
Server: CouchDB/1.1.2a785d32f-git (Erlang OTP/R14B03)
{"couchdb":"Welcome","version":"1.1.2a785d32f-git"}

[INFO] Created DB named `db1'
[INFO] Uploaded 5000 documents via _bulk_docs
(....)
[INFO] Uploaded 5000 documents via _bulk_docs
Building view.
{"total_rows":500000,"offset":0,"rows":[
{"id":"0001c0a1-edcb-4dbc-aa9d-533c73d980cb","key":["dwarf","assassin"],"va=
lue":[{"x":62038.32,"y":105825.29},{"x":90713.13,"y":128570.97},{"x":43836.=
37,"y":80517.12},{"x":71610.97,"y":143739.99},{"x":86038.39,"y":84731.8}]}
]}

real	5m44.374s
user	0m0.008s
sys	0m0.010s


Disk model APPLE SSD TS128C, quad core machine, 8Gb of ram.


On Tue, Feb 28, 2012 at 5:17 AM, Jason Smith <jhs@apache.org> wrote:
> Forgive the clean new thread. Hopefully it will not remain so.
>
> If you can, would you please clone https://github.com/jhs/slow_couchdb
>
> And build whatever Erlangs and CouchDB checkouts you see fit, and run
> the test. For example:
>
> =C2=A0 =C2=A0docs=3D500000 ./bench.sh small_doc.tpl
>
> That should run the test and, God willing, upload the results to a
> couch in the cloud. We should be able to use that information to
> identify who you are, whether you are on SSD, what Erlang and Couch
> build, and how fast it ran. Modulo bugs.


--=20
Filipe David Manana,

"Reasonable men adapt themselves to the world.
=C2=A0Unreasonable men adapt the world to themselves.
=C2=A0That's why all progress depends on unreasonable men."