couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Lehnardt <...@apache.org>
Subject Re: Please report your indexing speed
Date Wed, 29 Feb 2012 00:56:46 GMT
One more report.

I got suspicious of the rather short runtimes, so I picked the
default_doc and ran it at 500k:

bench_R14B04_1.1.1_default_doc.tpl.log     2m19.139s
bench_R14B04_1.2.x_default_doc.tpl.log     2m18.875s

It seems to me that we need more variation in what we test,
more OSs, larger ddocs, like the one Stefan linked to. Can
anyone help providing this?

Cheers
Jan
-- 


On Feb 29, 2012, at 00:23 , Jan Lehnardt wrote:

> For Robert Newson, avoiding bulk inserts to populate the dbs:
> 
> bench_R14B04_1.1.1_default_doc.tpl.log     0m19.692s
> bench_R14B04_1.2.x_default_doc.tpl.log     0m17.033s
> 
> bench_R14B04_1.1.1_nested_6k.tpl.log       1m31.393s
> bench_R14B04_1.2.x_nested_6k.tpl.log       0m42.010s
> 
> bench_R14B04_1.1.1_small_doc.tpl.log       0m8.103s
> bench_R14B04_1.2.x_small_doc.tpl.log       0m10.597s
> 
> bench_R14B04_1.1.1_wow.tpl.log             0m33.944s
> bench_R14B04_1.2.x_wow.tpl.log             0m27.087s
> 
> (Just R14B04, full logs available on demand)
> 
> Cheers
> Jan
> -- 
> 
> 
> On Feb 28, 2012, at 23:09 , Jan Lehnardt wrote:
> 
>> Same story, but spinning disk, 5400rpm:
>> 
>> bench_R14B04_1.1.1_default_doc.tpl.log     0m19.175s
>> bench_R14B04_1.2.x_default_doc.tpl.log     0m16.821s
>> bench_R15B_1.1.1_default_doc.tpl.log       0m13.050s
>> bench_R15B_1.2.x_default_doc.tpl.log       0m13.292s
>> 
>> bench_R14B04_1.1.1_nested_6k.tpl.log       1m26.941s
>> bench_R14B04_1.2.x_nested_6k.tpl.log       0m39.178s
>> bench_R15B_1.1.1_nested_6k.tpl.log         0m47.766s
>> bench_R15B_1.2.x_nested_6k.tpl.log         0m31.697s
>> 
>> bench_R14B04_1.1.1_small_doc.tpl.log       1m19.851s
>> bench_R14B04_1.2.x_small_doc.tpl.log       1m43.057s
>> bench_R15B_1.1.1_small_doc.tpl.log         0m52.249s
>> bench_R15B_1.2.x_small_doc.tpl.log         1m8.195s
>> 
>> bench_R14B04_1.1.1_wow.tpl.log             0m29.589s
>> bench_R14B04_1.2.x_wow.tpl.log             0m24.867s
>> bench_R15B_1.1.1_wow.tpl.log               0m20.171s
>> bench_R15B_1.2.x_wow.tpl.log               0m18.800s
>> 
>> Full logs at http://jan.prima.de/slow_couch/rust/
>> 
>> Cheers
>> Jan
>> -- 
>> 
>> 
>> On Feb 28, 2012, at 21:22 , Jan Lehnardt wrote:
>> 
>>> 
>>> # tl;dr:
>>> 
>>> bench_R14B04_1.1.1_default_doc.tpl.log     0m18.749s
>>> bench_R14B04_1.2.x_default_doc.tpl.log     0m16.304s
>>> bench_R15B_1.1.1_default_doc.tpl.log       0m12.946s
>>> bench_R15B_1.2.x_default_doc.tpl.log       0m13.616s
>>> 
>>> bench_R14B04_1.1.1_nested_6k.tpl.log       1m27.267s
>>> bench_R14B04_1.2.x_nested_6k.tpl.log       0m37.910s
>>> bench_R15B_1.1.1_nested_6k.tpl.log         0m46.963s
>>> bench_R15B_1.2.x_nested_6k.tpl.log         0m33.011s
>>> 
>>> bench_R14B04_1.1.1_small_doc.tpl.log       1m17.212s
>>> bench_R14B04_1.2.x_small_doc.tpl.log       1m41.383s
>>> bench_R15B_1.1.1_small_doc.tpl.log         0m52.858s
>>> bench_R15B_1.2.x_small_doc.tpl.log         1m9.043s
>>> 
>>> bench_R14B04_1.1.1_wow.tpl.log             0m29.842s
>>> bench_R14B04_1.2.x_wow.tpl.log             0m24.178s
>>> bench_R15B_1.1.1_wow.tpl.log               0m20.493s
>>> bench_R15B_1.2.x_wow.tpl.log               0m19.584s
>>> 
>>> (Full logs at [5])
>>> 
>>> 
>>> # Description
>>> 
>>> All of these are on Mac OS X 10.7.3 on an SSD.
>>> 
>>> I'll be running the same set on spinning disk and then Robert N asked
>>> me to populate the DBs not using builk docs. Since that's gonna take
>>> a while, I'll probably run this overnight.
>>> 
>>> All of the results are generated by my fork of Jason's slow_couchdb[1]
>>> and Filipe's seatoncouch[2].
>>> 
>>> The changes I've made is have the small_doc test run with 500k instead
>>> of 50k docs, added .view files to match the tpl files in
>>> seatoncouch/templates/* so we can have similar views use the different
>>> doc structures.
>>> 
>>> I also added two scripts to orchestrate the above testing in a more
>>> automated fashion. It also allows you to run the full matrix yourself.
>>> All you need is set up homebrew allow `brew switch erlang R14B04` and
>>> R15B (which is controlled in matrix.sh[3]) and have a git checkout of the
>>> CouchDB sources that allow you to do `git checkout 1.1.1` or `1.2.x`
>>> (which is controlled in runner.sh[4], adjust the path to the git checkout
>>> there as well).
>>> 
>>> matrix.sh also allows you to specify which docs to run.
>>> 
>>> Please shout if you need any more info about this test run or how to
>>> run this yourself.
>>> 
>>> 
>>> # Analysis
>>> 
>>> Inconclusive, I'l like to run this on larger dbs in general to see if
>>> there are more differences that shake out and I've yet have to run this
>>> on a spinning disk let alone another OS* or more complex view functions
>>> or larger design docs (like the one Stefan had).
>>> 
>>> * It shouldn't be too much work to port slow_couchdb to other OSs, I'll
>>> definitely be looking into that, but we can do with every bit of help :)
>>> 
>>> So far, I'm happy to conclude that while there are definitely provable
>>> differences, that we can live with them.
>>> 
>>> Cheers
>>> Jan
>>> -- 
>>> 
>>> 
>>> [1]: https://github.com/janl/slow_couchdb
>>> [2]: https://github.com/janl/seatoncouch
>>> [3]: https://github.com/janl/slow_couchdb/blob/master/matrix.sh
>>> [4]: https://github.com/janl/slow_couchdb/blob/master/runner.sh
>>> [5]: http://jan.prima.de/slow_couch/ssd/
>>> 
>>> 
>>> On Feb 28, 2012, at 18:53 , Filipe David Manana wrote:
>>> 
>>>> Jason, repeated my last test with the 1Kb docs (
>>>> https://gist.github.com/1930804, map function
>>>> http://friendpaste.com/5C99aqXocN6N6H1BAYIigs ) to cover branch 1.1.x
>>>> as well. Here are the full results (also in
>>>> https://gist.github.com/1930807):
>>>> 
>>>> 
>>>> Before COUCHDB-1186
>>>> 
>>>> fdmanana 23:21:05 ~/git/hub/slow_couchdb (master)> docs=500000
>>>> batch=5000 ./bench.sh wow.tpl
>>>> Server: CouchDB/1.2.0a-a68a792-git (Erlang OTP/R14B03)
>>>> {"couchdb":"Welcome","version":"1.2.0a-a68a792-git"}
>>>> 
>>>> [INFO] Created DB named `db1'
>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>> (....)
>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>> Building view.
>>>> {"total_rows":500000,"offset":0,"rows":[
>>>> {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
>>>> ]}
>>>> 
>>>> real	5m6.676s
>>>> user	0m0.009s
>>>> sys	0m0.010s
>>>> 
>>>> 
>>>> After COUCHDB-1186
>>>> 
>>>> fdmanana 23:50:07 ~/git/hub/slow_couchdb (master)> docs=500000
>>>> batch=5000 ./bench.sh wow.tpl
>>>> Server: CouchDB/1.2.0a-f023052-git (Erlang OTP/R14B03)
>>>> {"couchdb":"Welcome","version":"1.2.0a-f023052-git"}
>>>> 
>>>> [INFO] Created DB named `db1'
>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>> (....)
>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>> Building view.
>>>> {"total_rows":500000,"offset":0,"rows":[
>>>> {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
>>>> ]}
>>>> 
>>>> real	5m1.395s
>>>> user	0m0.008s
>>>> sys	0m0.010s
>>>> 
>>>> 
>>>> After COUCHDB-1186 + better queueing patch
>>>> (http://friendpaste.com/178nPFgfyyeGf2vtNRpL0w)
>>>> 
>>>> fdmanana 00:14:25 ~/git/hub/slow_couchdb (master)> docs=500000
>>>> batch=5000 ./bench.sh wow.tpl
>>>> Server: CouchDB/1.2.0a-f023052-git (Erlang OTP/R14B03)
>>>> {"couchdb":"Welcome","version":"1.2.0a-f023052-git"}
>>>> 
>>>> [INFO] Created DB named `db1'
>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>> (....)
>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>> Building view.
>>>> {"total_rows":500000,"offset":0,"rows":[
>>>> {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
>>>> ]}
>>>> 
>>>> real	4m48.175s
>>>> user	0m0.008s
>>>> sys	0m0.009s
>>>> 
>>>> 
>>>> CouchDB branch 1.1.x
>>>> 
>>>> fdmanana 08:16:58 ~/git/hub/slow_couchdb (master)> docs=500000
>>>> batch=5000 ./bench.sh wow.tpl
>>>> Server: CouchDB/1.1.2a785d32f-git (Erlang OTP/R14B03)
>>>> {"couchdb":"Welcome","version":"1.1.2a785d32f-git"}
>>>> 
>>>> [INFO] Created DB named `db1'
>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>> (....)
>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>> Building view.
>>>> {"total_rows":500000,"offset":0,"rows":[
>>>> {"id":"0001c0a1-edcb-4dbc-aa9d-533c73d980cb","key":["dwarf","assassin"],"value":[{"x":62038.32,"y":105825.29},{"x":90713.13,"y":128570.97},{"x":43836.37,"y":80517.12},{"x":71610.97,"y":143739.99},{"x":86038.39,"y":84731.8}]}
>>>> ]}
>>>> 
>>>> real	5m44.374s
>>>> user	0m0.008s
>>>> sys	0m0.010s
>>>> 
>>>> 
>>>> Disk model APPLE SSD TS128C, quad core machine, 8Gb of ram.
>>>> 
>>>> 
>>>> 
>>>> On Tue, Feb 28, 2012 at 5:17 AM, Jason Smith <jhs@apache.org> wrote:
>>>>> Forgive the clean new thread. Hopefully it will not remain so.
>>>>> 
>>>>> If you can, would you please clone https://github.com/jhs/slow_couchdb
>>>>> 
>>>>> And build whatever Erlangs and CouchDB checkouts you see fit, and run
>>>>> the test. For example:
>>>>> 
>>>>> docs=500000 ./bench.sh small_doc.tpl
>>>>> 
>>>>> That should run the test and, God willing, upload the results to a
>>>>> couch in the cloud. We should be able to use that information to
>>>>> identify who you are, whether you are on SSD, what Erlang and Couch
>>>>> build, and how fast it ran. Modulo bugs.
>>>> 
>>>> 
>>>> 
>>>> -- 
>>>> Filipe David Manana,
>>>> 
>>>> "Reasonable men adapt themselves to the world.
>>>> Unreasonable men adapt the world to themselves.
>>>> That's why all progress depends on unreasonable men."
>>> 
>> 
> 


Mime
View raw message