From dev-return-21190-apmail-couchdb-dev-archive=couchdb.apache.org@couchdb.apache.org Wed Feb 29 17:12:59 2012 Return-Path: X-Original-To: apmail-couchdb-dev-archive@www.apache.org Delivered-To: apmail-couchdb-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 1CDEB9C9A for ; Wed, 29 Feb 2012 17:12:59 +0000 (UTC) Received: (qmail 98174 invoked by uid 500); 29 Feb 2012 17:12:58 -0000 Delivered-To: apmail-couchdb-dev-archive@couchdb.apache.org Received: (qmail 98126 invoked by uid 500); 29 Feb 2012 17:12:58 -0000 Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@couchdb.apache.org Delivered-To: mailing list dev@couchdb.apache.org Received: (qmail 98114 invoked by uid 99); 29 Feb 2012 17:12:58 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 29 Feb 2012 17:12:58 +0000 X-ASF-Spam-Status: No, hits=0.7 required=5.0 tests=SPF_NEUTRAL,TO_NO_BRKTS_PCNT X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [80.244.253.218] (HELO mail.traeumt.net) (80.244.253.218) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 29 Feb 2012 17:12:53 +0000 Received: from [10.0.0.10] (91-64-198-154-dynip.superkabel.de [91.64.198.154]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.traeumt.net (Postfix) with ESMTPSA id CA38C3CE5F for ; Wed, 29 Feb 2012 18:12:31 +0100 (CET) Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Apple Message framework v1257) Subject: Re: Please report your indexing speed From: Jan Lehnardt In-Reply-To: Date: Wed, 29 Feb 2012 18:12:31 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <4568D302-D9F2-49FE-9251-B280127F879D@apache.org> <9BB2567A-6723-43F6-AC17-15109C020EC0@apache.org> <8395A163-1E6C-470A-892D-844020D71F09@apache.org> To: dev@couchdb.apache.org X-Mailer: Apple Mail (2.1257) X-Virus-Checked: Checked by ClamAV on apache.org Mac OS X 10.7.3, 2.6Ghz i7 (2x2 cores), R14B04 SSD: 1.1.1: 0m31.337s 1.2.x: 0m21.229s Spinning disk, 5400rpm: 1.1.1: 0m32.807s 1.2.x: 0m21.172s Cheers Jan --=20 On Feb 29, 2012, at 16:03 , Robert Newson wrote: > I've produced a new script that reproduces the view regression. I > apologize in advance for exposing my awful Bash scripting abilities > (also my inability to write "pure" shell. Your "Bashism" is my "it > works"). >=20 > I get 0m56.521s for 1.1.x and 1m17.108s for 1.2.x. That is, 1.1.x > complete the same task in only 72% of the time that 1.2.x takes. >=20 > http://friendpaste.com/UI1OcECLEzR6i4D75LqQy >=20 > It would be very useful if others could reproduce (or contradict!) = this finding. >=20 > B. >=20 > On 29 February 2012 00:56, Jan Lehnardt wrote: >> One more report. >>=20 >> I got suspicious of the rather short runtimes, so I picked the >> default_doc and ran it at 500k: >>=20 >> bench_R14B04_1.1.1_default_doc.tpl.log 2m19.139s >> bench_R14B04_1.2.x_default_doc.tpl.log 2m18.875s >>=20 >> It seems to me that we need more variation in what we test, >> more OSs, larger ddocs, like the one Stefan linked to. Can >> anyone help providing this? >>=20 >> Cheers >> Jan >> -- >>=20 >>=20 >> On Feb 29, 2012, at 00:23 , Jan Lehnardt wrote: >>=20 >>> For Robert Newson, avoiding bulk inserts to populate the dbs: >>>=20 >>> bench_R14B04_1.1.1_default_doc.tpl.log 0m19.692s >>> bench_R14B04_1.2.x_default_doc.tpl.log 0m17.033s >>>=20 >>> bench_R14B04_1.1.1_nested_6k.tpl.log 1m31.393s >>> bench_R14B04_1.2.x_nested_6k.tpl.log 0m42.010s >>>=20 >>> bench_R14B04_1.1.1_small_doc.tpl.log 0m8.103s >>> bench_R14B04_1.2.x_small_doc.tpl.log 0m10.597s >>>=20 >>> bench_R14B04_1.1.1_wow.tpl.log 0m33.944s >>> bench_R14B04_1.2.x_wow.tpl.log 0m27.087s >>>=20 >>> (Just R14B04, full logs available on demand) >>>=20 >>> Cheers >>> Jan >>> -- >>>=20 >>>=20 >>> On Feb 28, 2012, at 23:09 , Jan Lehnardt wrote: >>>=20 >>>> Same story, but spinning disk, 5400rpm: >>>>=20 >>>> bench_R14B04_1.1.1_default_doc.tpl.log 0m19.175s >>>> bench_R14B04_1.2.x_default_doc.tpl.log 0m16.821s >>>> bench_R15B_1.1.1_default_doc.tpl.log 0m13.050s >>>> bench_R15B_1.2.x_default_doc.tpl.log 0m13.292s >>>>=20 >>>> bench_R14B04_1.1.1_nested_6k.tpl.log 1m26.941s >>>> bench_R14B04_1.2.x_nested_6k.tpl.log 0m39.178s >>>> bench_R15B_1.1.1_nested_6k.tpl.log 0m47.766s >>>> bench_R15B_1.2.x_nested_6k.tpl.log 0m31.697s >>>>=20 >>>> bench_R14B04_1.1.1_small_doc.tpl.log 1m19.851s >>>> bench_R14B04_1.2.x_small_doc.tpl.log 1m43.057s >>>> bench_R15B_1.1.1_small_doc.tpl.log 0m52.249s >>>> bench_R15B_1.2.x_small_doc.tpl.log 1m8.195s >>>>=20 >>>> bench_R14B04_1.1.1_wow.tpl.log 0m29.589s >>>> bench_R14B04_1.2.x_wow.tpl.log 0m24.867s >>>> bench_R15B_1.1.1_wow.tpl.log 0m20.171s >>>> bench_R15B_1.2.x_wow.tpl.log 0m18.800s >>>>=20 >>>> Full logs at http://jan.prima.de/slow_couch/rust/ >>>>=20 >>>> Cheers >>>> Jan >>>> -- >>>>=20 >>>>=20 >>>> On Feb 28, 2012, at 21:22 , Jan Lehnardt wrote: >>>>=20 >>>>>=20 >>>>> # tl;dr: >>>>>=20 >>>>> bench_R14B04_1.1.1_default_doc.tpl.log 0m18.749s >>>>> bench_R14B04_1.2.x_default_doc.tpl.log 0m16.304s >>>>> bench_R15B_1.1.1_default_doc.tpl.log 0m12.946s >>>>> bench_R15B_1.2.x_default_doc.tpl.log 0m13.616s >>>>>=20 >>>>> bench_R14B04_1.1.1_nested_6k.tpl.log 1m27.267s >>>>> bench_R14B04_1.2.x_nested_6k.tpl.log 0m37.910s >>>>> bench_R15B_1.1.1_nested_6k.tpl.log 0m46.963s >>>>> bench_R15B_1.2.x_nested_6k.tpl.log 0m33.011s >>>>>=20 >>>>> bench_R14B04_1.1.1_small_doc.tpl.log 1m17.212s >>>>> bench_R14B04_1.2.x_small_doc.tpl.log 1m41.383s >>>>> bench_R15B_1.1.1_small_doc.tpl.log 0m52.858s >>>>> bench_R15B_1.2.x_small_doc.tpl.log 1m9.043s >>>>>=20 >>>>> bench_R14B04_1.1.1_wow.tpl.log 0m29.842s >>>>> bench_R14B04_1.2.x_wow.tpl.log 0m24.178s >>>>> bench_R15B_1.1.1_wow.tpl.log 0m20.493s >>>>> bench_R15B_1.2.x_wow.tpl.log 0m19.584s >>>>>=20 >>>>> (Full logs at [5]) >>>>>=20 >>>>>=20 >>>>> # Description >>>>>=20 >>>>> All of these are on Mac OS X 10.7.3 on an SSD. >>>>>=20 >>>>> I'll be running the same set on spinning disk and then Robert N = asked >>>>> me to populate the DBs not using builk docs. Since that's gonna = take >>>>> a while, I'll probably run this overnight. >>>>>=20 >>>>> All of the results are generated by my fork of Jason's = slow_couchdb[1] >>>>> and Filipe's seatoncouch[2]. >>>>>=20 >>>>> The changes I've made is have the small_doc test run with 500k = instead >>>>> of 50k docs, added .view files to match the tpl files in >>>>> seatoncouch/templates/* so we can have similar views use the = different >>>>> doc structures. >>>>>=20 >>>>> I also added two scripts to orchestrate the above testing in a = more >>>>> automated fashion. It also allows you to run the full matrix = yourself. >>>>> All you need is set up homebrew allow `brew switch erlang R14B04` = and >>>>> R15B (which is controlled in matrix.sh[3]) and have a git checkout = of the >>>>> CouchDB sources that allow you to do `git checkout 1.1.1` or = `1.2.x` >>>>> (which is controlled in runner.sh[4], adjust the path to the git = checkout >>>>> there as well). >>>>>=20 >>>>> matrix.sh also allows you to specify which docs to run. >>>>>=20 >>>>> Please shout if you need any more info about this test run or how = to >>>>> run this yourself. >>>>>=20 >>>>>=20 >>>>> # Analysis >>>>>=20 >>>>> Inconclusive, I'l like to run this on larger dbs in general to see = if >>>>> there are more differences that shake out and I've yet have to run = this >>>>> on a spinning disk let alone another OS* or more complex view = functions >>>>> or larger design docs (like the one Stefan had). >>>>>=20 >>>>> * It shouldn't be too much work to port slow_couchdb to other OSs, = I'll >>>>> definitely be looking into that, but we can do with every bit of = help :) >>>>>=20 >>>>> So far, I'm happy to conclude that while there are definitely = provable >>>>> differences, that we can live with them. >>>>>=20 >>>>> Cheers >>>>> Jan >>>>> -- >>>>>=20 >>>>>=20 >>>>> [1]: https://github.com/janl/slow_couchdb >>>>> [2]: https://github.com/janl/seatoncouch >>>>> [3]: https://github.com/janl/slow_couchdb/blob/master/matrix.sh >>>>> [4]: https://github.com/janl/slow_couchdb/blob/master/runner.sh >>>>> [5]: http://jan.prima.de/slow_couch/ssd/ >>>>>=20 >>>>>=20 >>>>> On Feb 28, 2012, at 18:53 , Filipe David Manana wrote: >>>>>=20 >>>>>> Jason, repeated my last test with the 1Kb docs ( >>>>>> https://gist.github.com/1930804, map function >>>>>> http://friendpaste.com/5C99aqXocN6N6H1BAYIigs ) to cover branch = 1.1.x >>>>>> as well. Here are the full results (also in >>>>>> https://gist.github.com/1930807): >>>>>>=20 >>>>>>=20 >>>>>> Before COUCHDB-1186 >>>>>>=20 >>>>>> fdmanana 23:21:05 ~/git/hub/slow_couchdb (master)> docs=3D500000 >>>>>> batch=3D5000 ./bench.sh wow.tpl >>>>>> Server: CouchDB/1.2.0a-a68a792-git (Erlang OTP/R14B03) >>>>>> {"couchdb":"Welcome","version":"1.2.0a-a68a792-git"} >>>>>>=20 >>>>>> [INFO] Created DB named `db1' >>>>>> [INFO] Uploaded 5000 documents via _bulk_docs >>>>>> (....) >>>>>> [INFO] Uploaded 5000 documents via _bulk_docs >>>>>> Building view. >>>>>> {"total_rows":500000,"offset":0,"rows":[ >>>>>> = {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"v= alue":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.1= 1,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}= >>>>>> ]} >>>>>>=20 >>>>>> real 5m6.676s >>>>>> user 0m0.009s >>>>>> sys 0m0.010s >>>>>>=20 >>>>>>=20 >>>>>> After COUCHDB-1186 >>>>>>=20 >>>>>> fdmanana 23:50:07 ~/git/hub/slow_couchdb (master)> docs=3D500000 >>>>>> batch=3D5000 ./bench.sh wow.tpl >>>>>> Server: CouchDB/1.2.0a-f023052-git (Erlang OTP/R14B03) >>>>>> {"couchdb":"Welcome","version":"1.2.0a-f023052-git"} >>>>>>=20 >>>>>> [INFO] Created DB named `db1' >>>>>> [INFO] Uploaded 5000 documents via _bulk_docs >>>>>> (....) >>>>>> [INFO] Uploaded 5000 documents via _bulk_docs >>>>>> Building view. >>>>>> {"total_rows":500000,"offset":0,"rows":[ >>>>>> = {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"v= alue":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.1= 1,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}= >>>>>> ]} >>>>>>=20 >>>>>> real 5m1.395s >>>>>> user 0m0.008s >>>>>> sys 0m0.010s >>>>>>=20 >>>>>>=20 >>>>>> After COUCHDB-1186 + better queueing patch >>>>>> (http://friendpaste.com/178nPFgfyyeGf2vtNRpL0w) >>>>>>=20 >>>>>> fdmanana 00:14:25 ~/git/hub/slow_couchdb (master)> docs=3D500000 >>>>>> batch=3D5000 ./bench.sh wow.tpl >>>>>> Server: CouchDB/1.2.0a-f023052-git (Erlang OTP/R14B03) >>>>>> {"couchdb":"Welcome","version":"1.2.0a-f023052-git"} >>>>>>=20 >>>>>> [INFO] Created DB named `db1' >>>>>> [INFO] Uploaded 5000 documents via _bulk_docs >>>>>> (....) >>>>>> [INFO] Uploaded 5000 documents via _bulk_docs >>>>>> Building view. >>>>>> {"total_rows":500000,"offset":0,"rows":[ >>>>>> = {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"v= alue":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.1= 1,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}= >>>>>> ]} >>>>>>=20 >>>>>> real 4m48.175s >>>>>> user 0m0.008s >>>>>> sys 0m0.009s >>>>>>=20 >>>>>>=20 >>>>>> CouchDB branch 1.1.x >>>>>>=20 >>>>>> fdmanana 08:16:58 ~/git/hub/slow_couchdb (master)> docs=3D500000 >>>>>> batch=3D5000 ./bench.sh wow.tpl >>>>>> Server: CouchDB/1.1.2a785d32f-git (Erlang OTP/R14B03) >>>>>> {"couchdb":"Welcome","version":"1.1.2a785d32f-git"} >>>>>>=20 >>>>>> [INFO] Created DB named `db1' >>>>>> [INFO] Uploaded 5000 documents via _bulk_docs >>>>>> (....) >>>>>> [INFO] Uploaded 5000 documents via _bulk_docs >>>>>> Building view. >>>>>> {"total_rows":500000,"offset":0,"rows":[ >>>>>> = {"id":"0001c0a1-edcb-4dbc-aa9d-533c73d980cb","key":["dwarf","assassin"],"v= alue":[{"x":62038.32,"y":105825.29},{"x":90713.13,"y":128570.97},{"x":4383= 6.37,"y":80517.12},{"x":71610.97,"y":143739.99},{"x":86038.39,"y":84731.8}= ]} >>>>>> ]} >>>>>>=20 >>>>>> real 5m44.374s >>>>>> user 0m0.008s >>>>>> sys 0m0.010s >>>>>>=20 >>>>>>=20 >>>>>> Disk model APPLE SSD TS128C, quad core machine, 8Gb of ram. >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>> On Tue, Feb 28, 2012 at 5:17 AM, Jason Smith = wrote: >>>>>>> Forgive the clean new thread. Hopefully it will not remain so. >>>>>>>=20 >>>>>>> If you can, would you please clone = https://github.com/jhs/slow_couchdb >>>>>>>=20 >>>>>>> And build whatever Erlangs and CouchDB checkouts you see fit, = and run >>>>>>> the test. For example: >>>>>>>=20 >>>>>>> docs=3D500000 ./bench.sh small_doc.tpl >>>>>>>=20 >>>>>>> That should run the test and, God willing, upload the results to = a >>>>>>> couch in the cloud. We should be able to use that information to >>>>>>> identify who you are, whether you are on SSD, what Erlang and = Couch >>>>>>> build, and how fast it ran. Modulo bugs. >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>> -- >>>>>> Filipe David Manana, >>>>>>=20 >>>>>> "Reasonable men adapt themselves to the world. >>>>>> Unreasonable men adapt the world to themselves. >>>>>> That's why all progress depends on unreasonable men." >>>>>=20 >>>>=20 >>>=20 >>=20