couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Davis <paul.joseph.da...@gmail.com>
Subject Re: There is no spoon.
Date Fri, 17 Jul 2009 16:17:55 GMT
On Fri, Jul 17, 2009 at 11:58 AM, Chris Anderson<jchris@apache.org> wrote:
> On Fri, Jul 17, 2009 at 12:35 AM, Paul Davis<paul.joseph.davis@gmail.com> wrote:
>> Hiya,
>>
>> I had me an idea the other day I got around to trying. We've been
>> going over how to make JSON parsing ├╝ber fast between Erlang and the
>> View servers. Instead of making JSON parsing faster I decided to just
>> drop it completely. I wrote enough code in couch_js.c tonight to get
>> the basics of converting the ErlJSON -> Spidermonkey objects and back.
>> Quite a few of the pertinent tests are passing. There was an issue
>> with object iteration that prevent view collation from working
>> correctly. And the show/list tests are broken because I didn't add XML
>> serialization. Either way it was enough for me to collect some numbers
>> with the same script I used on my blog a couple weeks ago.
>>
>> The huge ass caveat on the tail end for Patch at 10K 8KiB docs is that
>> this is best case scenario. I was just beefing up the document size by
>> adding a large string on them. In the conversion process this ends up
>> being a fairly quick pass using EncodeString and DecodeString.
>>
>> Another thing to notice is that once compiled with +native the numbers
>> for small docs don't change too drastically.
>>
>> And here are numbers:
>>
>> Straight up trunk: 10K tiny docs
>> --------------------------------
>>
>> Inserting: 0.955830
>> Map only: 4.296859
>> With reduce: 4.014233
>> With erlang reduce: 3.199325
>>
>> Inserting: 0.970745
>> Map only: 3.961110
>> With reduce: 4.550082
>> With erlang reduce: 3.493316
>>
>> Inserting: 0.992892
>> Map only: 4.747793
>> With reduce: 4.552446
>> With erlang reduce: 3.681820
>>
>>
>> Straight up trunk: 1OK 4KiB Documents
>> -------------------------------------
>>
>> Inserting: 5.895689
>> Map only: 11.716073
>> With reduce: 12.127348
>> With erlang reduce: 11.069352
>>
>> Inserting: 6.221656
>> Map only: 12.074525
>> With reduce: 11.500115
>> With erlang reduce: 10.680610
>>
>> Inserting: 5.974915
>> Map only: 11.240969
>> With reduce: 11.620035
>> With erlang reduce: 10.458795
>>
>>
>> Straight up trunk: 10K 8KiB Documents
>> -------------------------------------
>>
>> Inserting: 9.533340
>> Map only: 16.273873
>> With reduce: 16.647050
>> With erlang reduce: 14.529038
>>
>> Inserting: 9.828476
>> Map only: 15.772620
>> With reduce: 15.707862
>> With erlang reduce: 14.577865
>>
>> Inserting: 9.598872
>> Map only: 15.251671
>> With reduce: 15.930784
>> With erlang reduce: 14.445052
>>
>> Trunk +native 10K Tiny docs
>> ---------------------------
>>
>> Inserting: 0.953937
>> Map only: 2.524961
>> With reduce: 2.411511
>> With erlang reduce: 1.541173
>>
>> Inserting: 0.963175
>> Map only: 2.486752
>> With reduce: 2.354808
>> With erlang reduce: 1.534005
>>
>> Inserting: 0.949138
>> Map only: 2.429267
>> With reduce: 2.385016
>> With erlang reduce: 1.525428
>>
>>
>> Trunk +native 10K 4KiB docs
>> ---------------------------
>>
>> Inserting: 3.952355
>> Map only: 10.106112
>> With reduce: 9.687787
>> With erlang reduce: 8.781025
>>
>> Inserting: 3.968877
>> Map only: 9.552732
>> With reduce: 9.626942
>> With erlang reduce: 8.537417
>>
>> Inserting: 4.359648
>> Map only: 9.472417
>> With reduce: 9.719609
>> With erlang reduce: 8.771725
>>
>>
>> Trunk +native 10K 8KiB docs
>> ---------------------------
>>
>> Inserting: 7.046171
>> Map only: 12.111946
>> With reduce: 11.566371
>> With erlang reduce: 10.571792
>>
>> Inserting: 7.183114
>> Map only: 12.177807
>> With reduce: 11.619149
>> With erlang reduce: 10.461091
>>
>> Inserting: 6.867450
>> Map only: 11.358312
>> With reduce: 11.420452
>> With erlang reduce: 10.452706
>>
>>
>> Patched 10K Tiny docs
>> ---------------------
>>
>> Inserting: 0.954482
>> Map only: 2.339038
>> With reduce: 2.311544
>> With erlang reduce: 1.513258
>>
>> Inserting: 0.942735
>> Map only: 2.543295
>> With reduce: 2.522470
>> With erlang reduce: 1.514119
>>
>> Inserting: 0.961381
>> Map only: 2.372250
>> With reduce: 2.336503
>> With erlang reduce: 1.558217
>>
>>
>> Patched 10K 4KiB docs
>> ---------------------
>>
>> Inserting: 5.933259
>> Map only: 5.484083
>> With reduce: 5.693180
>> With erlang reduce: 4.502828
>>
>> Inserting: 5.980323
>> Map only: 5.251158
>> With reduce: 5.290837
>> With erlang reduce: 4.530348
>>
>> Inserting: 6.067070
>> Map only: 5.501945
>> With reduce: 5.314363
>> With erlang reduce: 4.409588
>>
>> Patched 10K 8KiB docs
>> ---------------------
>>
>> Inserting: 7.096909
>> Map only: 5.293864
>> With reduce: 5.254415
>> With erlang reduce: 4.437001
>>
>> Inserting: 6.847729
>> Map only: 5.191201
>> With reduce: 5.161696
>> With erlang reduce: 4.256955
>>
>> Inserting: 7.168672
>> Map only: 5.294789
>> With reduce: 5.195616
>> With erlang reduce: 4.323075
>>
>> Patched +native 10K Tiny docs
>> -----------------------------
>>
>> Inserting: 0.945826
>> Map only: 2.451693
>> With reduce: 2.383522
>> With erlang reduce: 1.578508
>>
>> Inserting: 0.972655
>> Map only: 2.504448
>> With reduce: 2.343594
>> With erlang reduce: 1.512512
>>
>> Inserting: 0.952105
>> Map only: 2.391866
>> With reduce: 2.329651
>> With erlang reduce: 1.505249
>>
>>
>> Patched +native 10K 4KiB docs
>> -----------------------------
>>
>> Inserting: 4.060498
>> Map only: 5.937243
>> With reduce: 5.611110
>> With erlang reduce: 4.813867
>>
>> Inserting: 3.979680
>> Map only: 5.720481
>> With reduce: 5.602648
>> With erlang reduce: 4.734298
>>
>> Inserting: 3.892947
>> Map only: 6.140995
>> With reduce: 5.565891
>> With erlang reduce: 4.736162
>>
>>
>> Patched +native 10K 8KiB docs
>> -----------------------------
>>
>> Inserting: 7.040456
>> Map only: 5.302867
>> With reduce: 5.269647
>> With erlang reduce: 4.433941
>>
>> Inserting: 6.808467
>> Map only: 5.371900
>> With reduce: 5.203042
>> With erlang reduce: 4.347747
>>
>> Inserting: 6.977694
>> Map only: 5.393854
>> With reduce: 5.368036
>> With erlang reduce: 4.368916
>>
>
> 3x faster+ is pretty good.
>
> Making JSON faster pays off for lots of languages, but if this is
> reliable and transparent to the JavaScript developer... it's tempting.
>

I could implement it blind for Python, and Ruby's got Erlectricity.
Its really pretty easy to implement once you figure out the basics of
what's going on. Right now the next part would be to figure out how to
get jsapi to mimic the JS VM which I've had issues with before with
python-spidermonkey. And I was thinking that if it did get kept it'd
be configurable for each view server. If you peep the actual changes
inside couch_query_servers.erl its pretty minimal. Basically just pass
slightly different port options and a different Read/Write function
pair.

Either way, it'll take some more work to see if the slight wrinkles
can be ironed out.

Paul Davis

> --
> Chris Anderson
> http://jchrisa.net
> http://couch.io
>

Mime
View raw message