Return-Path: Delivered-To: apmail-couchdb-dev-archive@www.apache.org Received: (qmail 23465 invoked from network); 17 Jul 2009 16:17:21 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 17 Jul 2009 16:17:21 -0000 Received: (qmail 56895 invoked by uid 500); 17 Jul 2009 16:18:26 -0000 Delivered-To: apmail-couchdb-dev-archive@couchdb.apache.org Received: (qmail 56820 invoked by uid 500); 17 Jul 2009 16:18:26 -0000 Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@couchdb.apache.org Delivered-To: mailing list dev@couchdb.apache.org Received: (qmail 56810 invoked by uid 99); 17 Jul 2009 16:18:26 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 17 Jul 2009 16:18:26 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of paul.joseph.davis@gmail.com designates 209.85.132.240 as permitted sender) Received: from [209.85.132.240] (HELO an-out-0708.google.com) (209.85.132.240) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 17 Jul 2009 16:18:16 +0000 Received: by an-out-0708.google.com with SMTP id b2so436537ana.5 for ; Fri, 17 Jul 2009 09:17:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=dfx4fJik4uUET9sp6+1o3ixT6WOEMwlg7H2vDm6ehnE=; b=rlfq5AIzqg6+hlCu2PISd0E2vN0pmAnxKGc31vHqQV5WYglrOjGhhgetUu+k+ZKN3S XOowSDdjH2Ooa5YJXJu+IviZTFNq8r6hrhq254VcTOJ7h7dl90i1bOhddB8hOjL5HlaF vNNVnyiW7C7pu0Oj54lCpDaeBpz8j9jYqG+gI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=bu8AHqgrm7GOPatGRw5pdVNTdKBeLiHsliQYhTGCSijJVF+K3YyydgH4Thw5Q4t69l z8qSGixxAfAhcEE/2NdlBdU7omAyj1FJaV3zv8qhXmDeEXk3e9nopvYVNYNA2eOmg+FW tFdXmnNo1UH/M8Lt27/KKdRkmJws6Vv1eZ6OY= MIME-Version: 1.0 Received: by 10.100.126.19 with SMTP id y19mr1773490anc.100.1247847475530; Fri, 17 Jul 2009 09:17:55 -0700 (PDT) In-Reply-To: References: Date: Fri, 17 Jul 2009 12:17:55 -0400 Message-ID: Subject: Re: There is no spoon. From: Paul Davis To: dev@couchdb.apache.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org On Fri, Jul 17, 2009 at 11:58 AM, Chris Anderson wrote: > On Fri, Jul 17, 2009 at 12:35 AM, Paul Davis= wrote: >> Hiya, >> >> I had me an idea the other day I got around to trying. We've been >> going over how to make JSON parsing =FCber fast between Erlang and the >> View servers. Instead of making JSON parsing faster I decided to just >> drop it completely. I wrote enough code in couch_js.c tonight to get >> the basics of converting the ErlJSON -> Spidermonkey objects and back. >> Quite a few of the pertinent tests are passing. There was an issue >> with object iteration that prevent view collation from working >> correctly. And the show/list tests are broken because I didn't add XML >> serialization. Either way it was enough for me to collect some numbers >> with the same script I used on my blog a couple weeks ago. >> >> The huge ass caveat on the tail end for Patch at 10K 8KiB docs is that >> this is best case scenario. I was just beefing up the document size by >> adding a large string on them. In the conversion process this ends up >> being a fairly quick pass using EncodeString and DecodeString. >> >> Another thing to notice is that once compiled with +native the numbers >> for small docs don't change too drastically. >> >> And here are numbers: >> >> Straight up trunk: 10K tiny docs >> -------------------------------- >> >> Inserting: 0.955830 >> Map only: 4.296859 >> With reduce: 4.014233 >> With erlang reduce: 3.199325 >> >> Inserting: 0.970745 >> Map only: 3.961110 >> With reduce: 4.550082 >> With erlang reduce: 3.493316 >> >> Inserting: 0.992892 >> Map only: 4.747793 >> With reduce: 4.552446 >> With erlang reduce: 3.681820 >> >> >> Straight up trunk: 1OK 4KiB Documents >> ------------------------------------- >> >> Inserting: 5.895689 >> Map only: 11.716073 >> With reduce: 12.127348 >> With erlang reduce: 11.069352 >> >> Inserting: 6.221656 >> Map only: 12.074525 >> With reduce: 11.500115 >> With erlang reduce: 10.680610 >> >> Inserting: 5.974915 >> Map only: 11.240969 >> With reduce: 11.620035 >> With erlang reduce: 10.458795 >> >> >> Straight up trunk: 10K 8KiB Documents >> ------------------------------------- >> >> Inserting: 9.533340 >> Map only: 16.273873 >> With reduce: 16.647050 >> With erlang reduce: 14.529038 >> >> Inserting: 9.828476 >> Map only: 15.772620 >> With reduce: 15.707862 >> With erlang reduce: 14.577865 >> >> Inserting: 9.598872 >> Map only: 15.251671 >> With reduce: 15.930784 >> With erlang reduce: 14.445052 >> >> Trunk +native 10K Tiny docs >> --------------------------- >> >> Inserting: 0.953937 >> Map only: 2.524961 >> With reduce: 2.411511 >> With erlang reduce: 1.541173 >> >> Inserting: 0.963175 >> Map only: 2.486752 >> With reduce: 2.354808 >> With erlang reduce: 1.534005 >> >> Inserting: 0.949138 >> Map only: 2.429267 >> With reduce: 2.385016 >> With erlang reduce: 1.525428 >> >> >> Trunk +native 10K 4KiB docs >> --------------------------- >> >> Inserting: 3.952355 >> Map only: 10.106112 >> With reduce: 9.687787 >> With erlang reduce: 8.781025 >> >> Inserting: 3.968877 >> Map only: 9.552732 >> With reduce: 9.626942 >> With erlang reduce: 8.537417 >> >> Inserting: 4.359648 >> Map only: 9.472417 >> With reduce: 9.719609 >> With erlang reduce: 8.771725 >> >> >> Trunk +native 10K 8KiB docs >> --------------------------- >> >> Inserting: 7.046171 >> Map only: 12.111946 >> With reduce: 11.566371 >> With erlang reduce: 10.571792 >> >> Inserting: 7.183114 >> Map only: 12.177807 >> With reduce: 11.619149 >> With erlang reduce: 10.461091 >> >> Inserting: 6.867450 >> Map only: 11.358312 >> With reduce: 11.420452 >> With erlang reduce: 10.452706 >> >> >> Patched 10K Tiny docs >> --------------------- >> >> Inserting: 0.954482 >> Map only: 2.339038 >> With reduce: 2.311544 >> With erlang reduce: 1.513258 >> >> Inserting: 0.942735 >> Map only: 2.543295 >> With reduce: 2.522470 >> With erlang reduce: 1.514119 >> >> Inserting: 0.961381 >> Map only: 2.372250 >> With reduce: 2.336503 >> With erlang reduce: 1.558217 >> >> >> Patched 10K 4KiB docs >> --------------------- >> >> Inserting: 5.933259 >> Map only: 5.484083 >> With reduce: 5.693180 >> With erlang reduce: 4.502828 >> >> Inserting: 5.980323 >> Map only: 5.251158 >> With reduce: 5.290837 >> With erlang reduce: 4.530348 >> >> Inserting: 6.067070 >> Map only: 5.501945 >> With reduce: 5.314363 >> With erlang reduce: 4.409588 >> >> Patched 10K 8KiB docs >> --------------------- >> >> Inserting: 7.096909 >> Map only: 5.293864 >> With reduce: 5.254415 >> With erlang reduce: 4.437001 >> >> Inserting: 6.847729 >> Map only: 5.191201 >> With reduce: 5.161696 >> With erlang reduce: 4.256955 >> >> Inserting: 7.168672 >> Map only: 5.294789 >> With reduce: 5.195616 >> With erlang reduce: 4.323075 >> >> Patched +native 10K Tiny docs >> ----------------------------- >> >> Inserting: 0.945826 >> Map only: 2.451693 >> With reduce: 2.383522 >> With erlang reduce: 1.578508 >> >> Inserting: 0.972655 >> Map only: 2.504448 >> With reduce: 2.343594 >> With erlang reduce: 1.512512 >> >> Inserting: 0.952105 >> Map only: 2.391866 >> With reduce: 2.329651 >> With erlang reduce: 1.505249 >> >> >> Patched +native 10K 4KiB docs >> ----------------------------- >> >> Inserting: 4.060498 >> Map only: 5.937243 >> With reduce: 5.611110 >> With erlang reduce: 4.813867 >> >> Inserting: 3.979680 >> Map only: 5.720481 >> With reduce: 5.602648 >> With erlang reduce: 4.734298 >> >> Inserting: 3.892947 >> Map only: 6.140995 >> With reduce: 5.565891 >> With erlang reduce: 4.736162 >> >> >> Patched +native 10K 8KiB docs >> ----------------------------- >> >> Inserting: 7.040456 >> Map only: 5.302867 >> With reduce: 5.269647 >> With erlang reduce: 4.433941 >> >> Inserting: 6.808467 >> Map only: 5.371900 >> With reduce: 5.203042 >> With erlang reduce: 4.347747 >> >> Inserting: 6.977694 >> Map only: 5.393854 >> With reduce: 5.368036 >> With erlang reduce: 4.368916 >> > > 3x faster+ is pretty good. > > Making JSON faster pays off for lots of languages, but if this is > reliable and transparent to the JavaScript developer... it's tempting. > I could implement it blind for Python, and Ruby's got Erlectricity. Its really pretty easy to implement once you figure out the basics of what's going on. Right now the next part would be to figure out how to get jsapi to mimic the JS VM which I've had issues with before with python-spidermonkey. And I was thinking that if it did get kept it'd be configurable for each view server. If you peep the actual changes inside couch_query_servers.erl its pretty minimal. Basically just pass slightly different port options and a different Read/Write function pair. Either way, it'll take some more work to see if the slight wrinkles can be ironed out. Paul Davis > -- > Chris Anderson > http://jchrisa.net > http://couch.io >