Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
Received-SPF: pass (nike.apache.org: domain of fdmanana@gmail.com designates
 209.85.212.172 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:sender:in-reply-to:references:date
         :x-google-sender-auth:message-id:subject:from:to:content-type
         :content-transfer-encoding;
        b=kUf0Qq/YeAbh5Ijfz55Zn0Q2K9XeHchw2ffEmAMYcy0dceZJl7mJbCDfLzE75TzDvW
         IrWBUA0QrSZE5rzyLdO6iAURkExXHUdCkKT7wnxeufBnExr9dQHVfv6iWtWyriOEFAhG
         RmOPxrlXijFKsB07Ny6q2tRZ1p7p7UcEQs4WI=
MIME-Version: 1.0
Sender: fdmanana@gmail.com
In-Reply-To: <BANLkTinqDXu=3x-XHfSdcF_sNTzimVdkrw@mail.gmail.com>
References: <BANLkTinqDXu=3x-XHfSdcF_sNTzimVdkrw@mail.gmail.com>
Date: Wed, 1 Jun 2011 13:34:44 +0100
Message-ID: <BANLkTi=Y7G2Vz0ymx1cYMMDrZb4BJVRjWQ@mail.gmail.com>
Subject: Re: Replacing ejson with jiffy
From: Filipe David Manana <fdmanana@apache.org>
To: dev@couchdb.apache.org
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On Sat, May 28, 2011 at 8:38 PM, Paul Davis <paul.joseph.davis@gmail.com> w=
rote:
> Recently we added the ejson NIF to trunk to speedup JSON parsing. I've
> finally put some finishing touches on a NIF that does much more of the
> actual encoding and decoding in C while still maintaining the same
> semantics as mochijson2. Specifically large numbers are still handled
> without truncation among a few other things like unicode handling.

Good work Paul.

>
> Things left to consider: I need to fall back to mochijson2 when jiffy
> isn't available due to having an old Erlang.
>
> More importantly, Jiffy requires an even newer Erlang than Ejson. I
> think R14B01, but it might even be R14B02. Not sure what people might
> think of that either.

Ideally I think R14B (< R14B01) should be supported. Is there really
some significant NIF API changes in 14B01 onwards that can not be
avoided? Anyway, I wouldn't make this a blocker.

>
> Thoughts?
>

Can you share your benchmark results, compared to latest trunk, and
how have you tested it?

I made a few basic tests today only for database writes only and view
indexing. I used couch_http_writes.sh and couch_http_bulk_writes.sh
from my github [1], both with 1Kb documents. I did not have the time
yet to see what relaximation results are.
Here they are:

HTTP api, single document writes, 50 writers, 5 minutes run
(couch_http_writes.sh)

     jiffy

     76 199 documents written in 5 minutes

     trunk

     79 513 documents written in 5 minutes


HTTP api, bulk writes, 50 writers, batches of 10 docs, 5 minutes run
(couch_http_bulk_writes.sh)

     jiffy

     616 960 documents written in 5 minutes

     trunk

     652 760 documents written in 5 minutes


HTTP api, bulk writes, 50 writers, batches of 100 docs, 5 minutes run
(couch_http_bulk_writes.sh)

     jiffy

     1 531 400 documents written in 5 minutes

     trunk

     1 314 200 documents written in 5 minutes


Indexing Beno=C3=AEt's warlogs database (https://warlogs.upondata.com/warlo=
gs):

$ time curl http://localhost:5984/warlogs/_design/warlogs/_view/by_date?lim=
it=3D1
{"total_rows":391832,"offset":0,"rows":[
{"id":"0104D7FC-0219-4C16-8531-97C60A59C70C","key":["2004-01","0104D7FC-021=
9-4C16-8531-97C60A59C70C"],"value":null}
]}

real	34m39.351s
user	0m0.028s
sys	0m0.036s

Takes roughly the same as trunk, variations of a few minutes (1, 2, 3)
are normal for this view (in machine at least, and always after
clearing the caches with "echo 3 > /proc/sys/vm/drop_caches")


[1] - https://github.com/fdmanana/basho_bench_couch


--=20
Filipe David Manana,
fdmanana@gmail.com, fdmanana@apache.org

"Reasonable men adapt themselves to the world.
=C2=A0Unreasonable men adapt the world to themselves.
=C2=A0That's why all progress depends on unreasonable men."