couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Lehnardt <...@apache.org>
Subject Re: Please report your indexing speed
Date Wed, 29 Feb 2012 17:12:31 GMT
Mac OS X 10.7.3, 2.6Ghz i7 (2x2 cores), R14B04

SSD:
1.1.1: 0m31.337s
1.2.x: 0m21.229s

Spinning disk, 5400rpm:
1.1.1: 0m32.807s
1.2.x: 0m21.172s

Cheers
Jan
-- 




On Feb 29, 2012, at 16:03 , Robert Newson wrote:

> I've produced a new script that reproduces the view regression. I
> apologize in advance for exposing my awful Bash scripting abilities
> (also my inability to write "pure" shell. Your "Bashism" is my "it
> works").
> 
> I get 0m56.521s for 1.1.x and 1m17.108s for 1.2.x. That is, 1.1.x
> complete the same task in only 72% of the time that 1.2.x takes.
> 
> http://friendpaste.com/UI1OcECLEzR6i4D75LqQy
> 
> It would be very useful if others could reproduce (or contradict!) this finding.
> 
> B.
> 
> On 29 February 2012 00:56, Jan Lehnardt <jan@apache.org> wrote:
>> One more report.
>> 
>> I got suspicious of the rather short runtimes, so I picked the
>> default_doc and ran it at 500k:
>> 
>> bench_R14B04_1.1.1_default_doc.tpl.log     2m19.139s
>> bench_R14B04_1.2.x_default_doc.tpl.log     2m18.875s
>> 
>> It seems to me that we need more variation in what we test,
>> more OSs, larger ddocs, like the one Stefan linked to. Can
>> anyone help providing this?
>> 
>> Cheers
>> Jan
>> --
>> 
>> 
>> On Feb 29, 2012, at 00:23 , Jan Lehnardt wrote:
>> 
>>> For Robert Newson, avoiding bulk inserts to populate the dbs:
>>> 
>>> bench_R14B04_1.1.1_default_doc.tpl.log     0m19.692s
>>> bench_R14B04_1.2.x_default_doc.tpl.log     0m17.033s
>>> 
>>> bench_R14B04_1.1.1_nested_6k.tpl.log       1m31.393s
>>> bench_R14B04_1.2.x_nested_6k.tpl.log       0m42.010s
>>> 
>>> bench_R14B04_1.1.1_small_doc.tpl.log       0m8.103s
>>> bench_R14B04_1.2.x_small_doc.tpl.log       0m10.597s
>>> 
>>> bench_R14B04_1.1.1_wow.tpl.log             0m33.944s
>>> bench_R14B04_1.2.x_wow.tpl.log             0m27.087s
>>> 
>>> (Just R14B04, full logs available on demand)
>>> 
>>> Cheers
>>> Jan
>>> --
>>> 
>>> 
>>> On Feb 28, 2012, at 23:09 , Jan Lehnardt wrote:
>>> 
>>>> Same story, but spinning disk, 5400rpm:
>>>> 
>>>> bench_R14B04_1.1.1_default_doc.tpl.log     0m19.175s
>>>> bench_R14B04_1.2.x_default_doc.tpl.log     0m16.821s
>>>> bench_R15B_1.1.1_default_doc.tpl.log       0m13.050s
>>>> bench_R15B_1.2.x_default_doc.tpl.log       0m13.292s
>>>> 
>>>> bench_R14B04_1.1.1_nested_6k.tpl.log       1m26.941s
>>>> bench_R14B04_1.2.x_nested_6k.tpl.log       0m39.178s
>>>> bench_R15B_1.1.1_nested_6k.tpl.log         0m47.766s
>>>> bench_R15B_1.2.x_nested_6k.tpl.log         0m31.697s
>>>> 
>>>> bench_R14B04_1.1.1_small_doc.tpl.log       1m19.851s
>>>> bench_R14B04_1.2.x_small_doc.tpl.log       1m43.057s
>>>> bench_R15B_1.1.1_small_doc.tpl.log         0m52.249s
>>>> bench_R15B_1.2.x_small_doc.tpl.log         1m8.195s
>>>> 
>>>> bench_R14B04_1.1.1_wow.tpl.log             0m29.589s
>>>> bench_R14B04_1.2.x_wow.tpl.log             0m24.867s
>>>> bench_R15B_1.1.1_wow.tpl.log               0m20.171s
>>>> bench_R15B_1.2.x_wow.tpl.log               0m18.800s
>>>> 
>>>> Full logs at http://jan.prima.de/slow_couch/rust/
>>>> 
>>>> Cheers
>>>> Jan
>>>> --
>>>> 
>>>> 
>>>> On Feb 28, 2012, at 21:22 , Jan Lehnardt wrote:
>>>> 
>>>>> 
>>>>> # tl;dr:
>>>>> 
>>>>> bench_R14B04_1.1.1_default_doc.tpl.log     0m18.749s
>>>>> bench_R14B04_1.2.x_default_doc.tpl.log     0m16.304s
>>>>> bench_R15B_1.1.1_default_doc.tpl.log       0m12.946s
>>>>> bench_R15B_1.2.x_default_doc.tpl.log       0m13.616s
>>>>> 
>>>>> bench_R14B04_1.1.1_nested_6k.tpl.log       1m27.267s
>>>>> bench_R14B04_1.2.x_nested_6k.tpl.log       0m37.910s
>>>>> bench_R15B_1.1.1_nested_6k.tpl.log         0m46.963s
>>>>> bench_R15B_1.2.x_nested_6k.tpl.log         0m33.011s
>>>>> 
>>>>> bench_R14B04_1.1.1_small_doc.tpl.log       1m17.212s
>>>>> bench_R14B04_1.2.x_small_doc.tpl.log       1m41.383s
>>>>> bench_R15B_1.1.1_small_doc.tpl.log         0m52.858s
>>>>> bench_R15B_1.2.x_small_doc.tpl.log         1m9.043s
>>>>> 
>>>>> bench_R14B04_1.1.1_wow.tpl.log             0m29.842s
>>>>> bench_R14B04_1.2.x_wow.tpl.log             0m24.178s
>>>>> bench_R15B_1.1.1_wow.tpl.log               0m20.493s
>>>>> bench_R15B_1.2.x_wow.tpl.log               0m19.584s
>>>>> 
>>>>> (Full logs at [5])
>>>>> 
>>>>> 
>>>>> # Description
>>>>> 
>>>>> All of these are on Mac OS X 10.7.3 on an SSD.
>>>>> 
>>>>> I'll be running the same set on spinning disk and then Robert N asked
>>>>> me to populate the DBs not using builk docs. Since that's gonna take
>>>>> a while, I'll probably run this overnight.
>>>>> 
>>>>> All of the results are generated by my fork of Jason's slow_couchdb[1]
>>>>> and Filipe's seatoncouch[2].
>>>>> 
>>>>> The changes I've made is have the small_doc test run with 500k instead
>>>>> of 50k docs, added .view files to match the tpl files in
>>>>> seatoncouch/templates/* so we can have similar views use the different
>>>>> doc structures.
>>>>> 
>>>>> I also added two scripts to orchestrate the above testing in a more
>>>>> automated fashion. It also allows you to run the full matrix yourself.
>>>>> All you need is set up homebrew allow `brew switch erlang R14B04` and
>>>>> R15B (which is controlled in matrix.sh[3]) and have a git checkout of
the
>>>>> CouchDB sources that allow you to do `git checkout 1.1.1` or `1.2.x`
>>>>> (which is controlled in runner.sh[4], adjust the path to the git checkout
>>>>> there as well).
>>>>> 
>>>>> matrix.sh also allows you to specify which docs to run.
>>>>> 
>>>>> Please shout if you need any more info about this test run or how to
>>>>> run this yourself.
>>>>> 
>>>>> 
>>>>> # Analysis
>>>>> 
>>>>> Inconclusive, I'l like to run this on larger dbs in general to see if
>>>>> there are more differences that shake out and I've yet have to run this
>>>>> on a spinning disk let alone another OS* or more complex view functions
>>>>> or larger design docs (like the one Stefan had).
>>>>> 
>>>>> * It shouldn't be too much work to port slow_couchdb to other OSs, I'll
>>>>> definitely be looking into that, but we can do with every bit of help
:)
>>>>> 
>>>>> So far, I'm happy to conclude that while there are definitely provable
>>>>> differences, that we can live with them.
>>>>> 
>>>>> Cheers
>>>>> Jan
>>>>> --
>>>>> 
>>>>> 
>>>>> [1]: https://github.com/janl/slow_couchdb
>>>>> [2]: https://github.com/janl/seatoncouch
>>>>> [3]: https://github.com/janl/slow_couchdb/blob/master/matrix.sh
>>>>> [4]: https://github.com/janl/slow_couchdb/blob/master/runner.sh
>>>>> [5]: http://jan.prima.de/slow_couch/ssd/
>>>>> 
>>>>> 
>>>>> On Feb 28, 2012, at 18:53 , Filipe David Manana wrote:
>>>>> 
>>>>>> Jason, repeated my last test with the 1Kb docs (
>>>>>> https://gist.github.com/1930804, map function
>>>>>> http://friendpaste.com/5C99aqXocN6N6H1BAYIigs ) to cover branch 1.1.x
>>>>>> as well. Here are the full results (also in
>>>>>> https://gist.github.com/1930807):
>>>>>> 
>>>>>> 
>>>>>> Before COUCHDB-1186
>>>>>> 
>>>>>> fdmanana 23:21:05 ~/git/hub/slow_couchdb (master)> docs=500000
>>>>>> batch=5000 ./bench.sh wow.tpl
>>>>>> Server: CouchDB/1.2.0a-a68a792-git (Erlang OTP/R14B03)
>>>>>> {"couchdb":"Welcome","version":"1.2.0a-a68a792-git"}
>>>>>> 
>>>>>> [INFO] Created DB named `db1'
>>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>>> (....)
>>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>>> Building view.
>>>>>> {"total_rows":500000,"offset":0,"rows":[
>>>>>> {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
>>>>>> ]}
>>>>>> 
>>>>>> real       5m6.676s
>>>>>> user       0m0.009s
>>>>>> sys        0m0.010s
>>>>>> 
>>>>>> 
>>>>>> After COUCHDB-1186
>>>>>> 
>>>>>> fdmanana 23:50:07 ~/git/hub/slow_couchdb (master)> docs=500000
>>>>>> batch=5000 ./bench.sh wow.tpl
>>>>>> Server: CouchDB/1.2.0a-f023052-git (Erlang OTP/R14B03)
>>>>>> {"couchdb":"Welcome","version":"1.2.0a-f023052-git"}
>>>>>> 
>>>>>> [INFO] Created DB named `db1'
>>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>>> (....)
>>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>>> Building view.
>>>>>> {"total_rows":500000,"offset":0,"rows":[
>>>>>> {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
>>>>>> ]}
>>>>>> 
>>>>>> real       5m1.395s
>>>>>> user       0m0.008s
>>>>>> sys        0m0.010s
>>>>>> 
>>>>>> 
>>>>>> After COUCHDB-1186 + better queueing patch
>>>>>> (http://friendpaste.com/178nPFgfyyeGf2vtNRpL0w)
>>>>>> 
>>>>>> fdmanana 00:14:25 ~/git/hub/slow_couchdb (master)> docs=500000
>>>>>> batch=5000 ./bench.sh wow.tpl
>>>>>> Server: CouchDB/1.2.0a-f023052-git (Erlang OTP/R14B03)
>>>>>> {"couchdb":"Welcome","version":"1.2.0a-f023052-git"}
>>>>>> 
>>>>>> [INFO] Created DB named `db1'
>>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>>> (....)
>>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>>> Building view.
>>>>>> {"total_rows":500000,"offset":0,"rows":[
>>>>>> {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
>>>>>> ]}
>>>>>> 
>>>>>> real       4m48.175s
>>>>>> user       0m0.008s
>>>>>> sys        0m0.009s
>>>>>> 
>>>>>> 
>>>>>> CouchDB branch 1.1.x
>>>>>> 
>>>>>> fdmanana 08:16:58 ~/git/hub/slow_couchdb (master)> docs=500000
>>>>>> batch=5000 ./bench.sh wow.tpl
>>>>>> Server: CouchDB/1.1.2a785d32f-git (Erlang OTP/R14B03)
>>>>>> {"couchdb":"Welcome","version":"1.1.2a785d32f-git"}
>>>>>> 
>>>>>> [INFO] Created DB named `db1'
>>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>>> (....)
>>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>>> Building view.
>>>>>> {"total_rows":500000,"offset":0,"rows":[
>>>>>> {"id":"0001c0a1-edcb-4dbc-aa9d-533c73d980cb","key":["dwarf","assassin"],"value":[{"x":62038.32,"y":105825.29},{"x":90713.13,"y":128570.97},{"x":43836.37,"y":80517.12},{"x":71610.97,"y":143739.99},{"x":86038.39,"y":84731.8}]}
>>>>>> ]}
>>>>>> 
>>>>>> real       5m44.374s
>>>>>> user       0m0.008s
>>>>>> sys        0m0.010s
>>>>>> 
>>>>>> 
>>>>>> Disk model APPLE SSD TS128C, quad core machine, 8Gb of ram.
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> On Tue, Feb 28, 2012 at 5:17 AM, Jason Smith <jhs@apache.org>
wrote:
>>>>>>> Forgive the clean new thread. Hopefully it will not remain so.
>>>>>>> 
>>>>>>> If you can, would you please clone https://github.com/jhs/slow_couchdb
>>>>>>> 
>>>>>>> And build whatever Erlangs and CouchDB checkouts you see fit,
and run
>>>>>>> the test. For example:
>>>>>>> 
>>>>>>> docs=500000 ./bench.sh small_doc.tpl
>>>>>>> 
>>>>>>> That should run the test and, God willing, upload the results
to a
>>>>>>> couch in the cloud. We should be able to use that information
to
>>>>>>> identify who you are, whether you are on SSD, what Erlang and
Couch
>>>>>>> build, and how fast it ran. Modulo bugs.
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> --
>>>>>> Filipe David Manana,
>>>>>> 
>>>>>> "Reasonable men adapt themselves to the world.
>>>>>> Unreasonable men adapt the world to themselves.
>>>>>> That's why all progress depends on unreasonable men."
>>>>> 
>>>> 
>>> 
>> 


Mime
View raw message