couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Newson <rnew...@apache.org>
Subject Re: Please report your indexing speed
Date Wed, 29 Feb 2012 15:03:58 GMT
I've produced a new script that reproduces the view regression. I
apologize in advance for exposing my awful Bash scripting abilities
(also my inability to write "pure" shell. Your "Bashism" is my "it
works").

I get 0m56.521s for 1.1.x and 1m17.108s for 1.2.x. That is, 1.1.x
complete the same task in only 72% of the time that 1.2.x takes.

http://friendpaste.com/UI1OcECLEzR6i4D75LqQy

It would be very useful if others could reproduce (or contradict!) this finding.

B.

On 29 February 2012 00:56, Jan Lehnardt <jan@apache.org> wrote:
> One more report.
>
> I got suspicious of the rather short runtimes, so I picked the
> default_doc and ran it at 500k:
>
> bench_R14B04_1.1.1_default_doc.tpl.log     2m19.139s
> bench_R14B04_1.2.x_default_doc.tpl.log     2m18.875s
>
> It seems to me that we need more variation in what we test,
> more OSs, larger ddocs, like the one Stefan linked to. Can
> anyone help providing this?
>
> Cheers
> Jan
> --
>
>
> On Feb 29, 2012, at 00:23 , Jan Lehnardt wrote:
>
>> For Robert Newson, avoiding bulk inserts to populate the dbs:
>>
>> bench_R14B04_1.1.1_default_doc.tpl.log     0m19.692s
>> bench_R14B04_1.2.x_default_doc.tpl.log     0m17.033s
>>
>> bench_R14B04_1.1.1_nested_6k.tpl.log       1m31.393s
>> bench_R14B04_1.2.x_nested_6k.tpl.log       0m42.010s
>>
>> bench_R14B04_1.1.1_small_doc.tpl.log       0m8.103s
>> bench_R14B04_1.2.x_small_doc.tpl.log       0m10.597s
>>
>> bench_R14B04_1.1.1_wow.tpl.log             0m33.944s
>> bench_R14B04_1.2.x_wow.tpl.log             0m27.087s
>>
>> (Just R14B04, full logs available on demand)
>>
>> Cheers
>> Jan
>> --
>>
>>
>> On Feb 28, 2012, at 23:09 , Jan Lehnardt wrote:
>>
>>> Same story, but spinning disk, 5400rpm:
>>>
>>> bench_R14B04_1.1.1_default_doc.tpl.log     0m19.175s
>>> bench_R14B04_1.2.x_default_doc.tpl.log     0m16.821s
>>> bench_R15B_1.1.1_default_doc.tpl.log       0m13.050s
>>> bench_R15B_1.2.x_default_doc.tpl.log       0m13.292s
>>>
>>> bench_R14B04_1.1.1_nested_6k.tpl.log       1m26.941s
>>> bench_R14B04_1.2.x_nested_6k.tpl.log       0m39.178s
>>> bench_R15B_1.1.1_nested_6k.tpl.log         0m47.766s
>>> bench_R15B_1.2.x_nested_6k.tpl.log         0m31.697s
>>>
>>> bench_R14B04_1.1.1_small_doc.tpl.log       1m19.851s
>>> bench_R14B04_1.2.x_small_doc.tpl.log       1m43.057s
>>> bench_R15B_1.1.1_small_doc.tpl.log         0m52.249s
>>> bench_R15B_1.2.x_small_doc.tpl.log         1m8.195s
>>>
>>> bench_R14B04_1.1.1_wow.tpl.log             0m29.589s
>>> bench_R14B04_1.2.x_wow.tpl.log             0m24.867s
>>> bench_R15B_1.1.1_wow.tpl.log               0m20.171s
>>> bench_R15B_1.2.x_wow.tpl.log               0m18.800s
>>>
>>> Full logs at http://jan.prima.de/slow_couch/rust/
>>>
>>> Cheers
>>> Jan
>>> --
>>>
>>>
>>> On Feb 28, 2012, at 21:22 , Jan Lehnardt wrote:
>>>
>>>>
>>>> # tl;dr:
>>>>
>>>> bench_R14B04_1.1.1_default_doc.tpl.log     0m18.749s
>>>> bench_R14B04_1.2.x_default_doc.tpl.log     0m16.304s
>>>> bench_R15B_1.1.1_default_doc.tpl.log       0m12.946s
>>>> bench_R15B_1.2.x_default_doc.tpl.log       0m13.616s
>>>>
>>>> bench_R14B04_1.1.1_nested_6k.tpl.log       1m27.267s
>>>> bench_R14B04_1.2.x_nested_6k.tpl.log       0m37.910s
>>>> bench_R15B_1.1.1_nested_6k.tpl.log         0m46.963s
>>>> bench_R15B_1.2.x_nested_6k.tpl.log         0m33.011s
>>>>
>>>> bench_R14B04_1.1.1_small_doc.tpl.log       1m17.212s
>>>> bench_R14B04_1.2.x_small_doc.tpl.log       1m41.383s
>>>> bench_R15B_1.1.1_small_doc.tpl.log         0m52.858s
>>>> bench_R15B_1.2.x_small_doc.tpl.log         1m9.043s
>>>>
>>>> bench_R14B04_1.1.1_wow.tpl.log             0m29.842s
>>>> bench_R14B04_1.2.x_wow.tpl.log             0m24.178s
>>>> bench_R15B_1.1.1_wow.tpl.log               0m20.493s
>>>> bench_R15B_1.2.x_wow.tpl.log               0m19.584s
>>>>
>>>> (Full logs at [5])
>>>>
>>>>
>>>> # Description
>>>>
>>>> All of these are on Mac OS X 10.7.3 on an SSD.
>>>>
>>>> I'll be running the same set on spinning disk and then Robert N asked
>>>> me to populate the DBs not using builk docs. Since that's gonna take
>>>> a while, I'll probably run this overnight.
>>>>
>>>> All of the results are generated by my fork of Jason's slow_couchdb[1]
>>>> and Filipe's seatoncouch[2].
>>>>
>>>> The changes I've made is have the small_doc test run with 500k instead
>>>> of 50k docs, added .view files to match the tpl files in
>>>> seatoncouch/templates/* so we can have similar views use the different
>>>> doc structures.
>>>>
>>>> I also added two scripts to orchestrate the above testing in a more
>>>> automated fashion. It also allows you to run the full matrix yourself.
>>>> All you need is set up homebrew allow `brew switch erlang R14B04` and
>>>> R15B (which is controlled in matrix.sh[3]) and have a git checkout of the
>>>> CouchDB sources that allow you to do `git checkout 1.1.1` or `1.2.x`
>>>> (which is controlled in runner.sh[4], adjust the path to the git checkout
>>>> there as well).
>>>>
>>>> matrix.sh also allows you to specify which docs to run.
>>>>
>>>> Please shout if you need any more info about this test run or how to
>>>> run this yourself.
>>>>
>>>>
>>>> # Analysis
>>>>
>>>> Inconclusive, I'l like to run this on larger dbs in general to see if
>>>> there are more differences that shake out and I've yet have to run this
>>>> on a spinning disk let alone another OS* or more complex view functions
>>>> or larger design docs (like the one Stefan had).
>>>>
>>>> * It shouldn't be too much work to port slow_couchdb to other OSs, I'll
>>>> definitely be looking into that, but we can do with every bit of help :)
>>>>
>>>> So far, I'm happy to conclude that while there are definitely provable
>>>> differences, that we can live with them.
>>>>
>>>> Cheers
>>>> Jan
>>>> --
>>>>
>>>>
>>>> [1]: https://github.com/janl/slow_couchdb
>>>> [2]: https://github.com/janl/seatoncouch
>>>> [3]: https://github.com/janl/slow_couchdb/blob/master/matrix.sh
>>>> [4]: https://github.com/janl/slow_couchdb/blob/master/runner.sh
>>>> [5]: http://jan.prima.de/slow_couch/ssd/
>>>>
>>>>
>>>> On Feb 28, 2012, at 18:53 , Filipe David Manana wrote:
>>>>
>>>>> Jason, repeated my last test with the 1Kb docs (
>>>>> https://gist.github.com/1930804, map function
>>>>> http://friendpaste.com/5C99aqXocN6N6H1BAYIigs ) to cover branch 1.1.x
>>>>> as well. Here are the full results (also in
>>>>> https://gist.github.com/1930807):
>>>>>
>>>>>
>>>>> Before COUCHDB-1186
>>>>>
>>>>> fdmanana 23:21:05 ~/git/hub/slow_couchdb (master)> docs=500000
>>>>> batch=5000 ./bench.sh wow.tpl
>>>>> Server: CouchDB/1.2.0a-a68a792-git (Erlang OTP/R14B03)
>>>>> {"couchdb":"Welcome","version":"1.2.0a-a68a792-git"}
>>>>>
>>>>> [INFO] Created DB named `db1'
>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>> (....)
>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>> Building view.
>>>>> {"total_rows":500000,"offset":0,"rows":[
>>>>> {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
>>>>> ]}
>>>>>
>>>>> real       5m6.676s
>>>>> user       0m0.009s
>>>>> sys        0m0.010s
>>>>>
>>>>>
>>>>> After COUCHDB-1186
>>>>>
>>>>> fdmanana 23:50:07 ~/git/hub/slow_couchdb (master)> docs=500000
>>>>> batch=5000 ./bench.sh wow.tpl
>>>>> Server: CouchDB/1.2.0a-f023052-git (Erlang OTP/R14B03)
>>>>> {"couchdb":"Welcome","version":"1.2.0a-f023052-git"}
>>>>>
>>>>> [INFO] Created DB named `db1'
>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>> (....)
>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>> Building view.
>>>>> {"total_rows":500000,"offset":0,"rows":[
>>>>> {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
>>>>> ]}
>>>>>
>>>>> real       5m1.395s
>>>>> user       0m0.008s
>>>>> sys        0m0.010s
>>>>>
>>>>>
>>>>> After COUCHDB-1186 + better queueing patch
>>>>> (http://friendpaste.com/178nPFgfyyeGf2vtNRpL0w)
>>>>>
>>>>> fdmanana 00:14:25 ~/git/hub/slow_couchdb (master)> docs=500000
>>>>> batch=5000 ./bench.sh wow.tpl
>>>>> Server: CouchDB/1.2.0a-f023052-git (Erlang OTP/R14B03)
>>>>> {"couchdb":"Welcome","version":"1.2.0a-f023052-git"}
>>>>>
>>>>> [INFO] Created DB named `db1'
>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>> (....)
>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>> Building view.
>>>>> {"total_rows":500000,"offset":0,"rows":[
>>>>> {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
>>>>> ]}
>>>>>
>>>>> real       4m48.175s
>>>>> user       0m0.008s
>>>>> sys        0m0.009s
>>>>>
>>>>>
>>>>> CouchDB branch 1.1.x
>>>>>
>>>>> fdmanana 08:16:58 ~/git/hub/slow_couchdb (master)> docs=500000
>>>>> batch=5000 ./bench.sh wow.tpl
>>>>> Server: CouchDB/1.1.2a785d32f-git (Erlang OTP/R14B03)
>>>>> {"couchdb":"Welcome","version":"1.1.2a785d32f-git"}
>>>>>
>>>>> [INFO] Created DB named `db1'
>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>> (....)
>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>> Building view.
>>>>> {"total_rows":500000,"offset":0,"rows":[
>>>>> {"id":"0001c0a1-edcb-4dbc-aa9d-533c73d980cb","key":["dwarf","assassin"],"value":[{"x":62038.32,"y":105825.29},{"x":90713.13,"y":128570.97},{"x":43836.37,"y":80517.12},{"x":71610.97,"y":143739.99},{"x":86038.39,"y":84731.8}]}
>>>>> ]}
>>>>>
>>>>> real       5m44.374s
>>>>> user       0m0.008s
>>>>> sys        0m0.010s
>>>>>
>>>>>
>>>>> Disk model APPLE SSD TS128C, quad core machine, 8Gb of ram.
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Feb 28, 2012 at 5:17 AM, Jason Smith <jhs@apache.org> wrote:
>>>>>> Forgive the clean new thread. Hopefully it will not remain so.
>>>>>>
>>>>>> If you can, would you please clone https://github.com/jhs/slow_couchdb
>>>>>>
>>>>>> And build whatever Erlangs and CouchDB checkouts you see fit, and
run
>>>>>> the test. For example:
>>>>>>
>>>>>> docs=500000 ./bench.sh small_doc.tpl
>>>>>>
>>>>>> That should run the test and, God willing, upload the results to
a
>>>>>> couch in the cloud. We should be able to use that information to
>>>>>> identify who you are, whether you are on SSD, what Erlang and Couch
>>>>>> build, and how fast it ran. Modulo bugs.
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Filipe David Manana,
>>>>>
>>>>> "Reasonable men adapt themselves to the world.
>>>>> Unreasonable men adapt the world to themselves.
>>>>> That's why all progress depends on unreasonable men."
>>>>
>>>
>>
>

Mime
View raw message