couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Filipe David Manana <fdman...@apache.org>
Subject Re: Replacing ejson with jiffy
Date Wed, 01 Jun 2011 12:34:44 GMT
On Sat, May 28, 2011 at 8:38 PM, Paul Davis <paul.joseph.davis@gmail.com> wrote:
> Recently we added the ejson NIF to trunk to speedup JSON parsing. I've
> finally put some finishing touches on a NIF that does much more of the
> actual encoding and decoding in C while still maintaining the same
> semantics as mochijson2. Specifically large numbers are still handled
> without truncation among a few other things like unicode handling.

Good work Paul.

>
> Things left to consider: I need to fall back to mochijson2 when jiffy
> isn't available due to having an old Erlang.
>
> More importantly, Jiffy requires an even newer Erlang than Ejson. I
> think R14B01, but it might even be R14B02. Not sure what people might
> think of that either.

Ideally I think R14B (< R14B01) should be supported. Is there really
some significant NIF API changes in 14B01 onwards that can not be
avoided? Anyway, I wouldn't make this a blocker.

>
> Thoughts?
>

Can you share your benchmark results, compared to latest trunk, and
how have you tested it?

I made a few basic tests today only for database writes only and view
indexing. I used couch_http_writes.sh and couch_http_bulk_writes.sh
from my github [1], both with 1Kb documents. I did not have the time
yet to see what relaximation results are.
Here they are:

HTTP api, single document writes, 50 writers, 5 minutes run
(couch_http_writes.sh)

     jiffy

     76 199 documents written in 5 minutes

     trunk

     79 513 documents written in 5 minutes


HTTP api, bulk writes, 50 writers, batches of 10 docs, 5 minutes run
(couch_http_bulk_writes.sh)

     jiffy

     616 960 documents written in 5 minutes

     trunk

     652 760 documents written in 5 minutes


HTTP api, bulk writes, 50 writers, batches of 100 docs, 5 minutes run
(couch_http_bulk_writes.sh)

     jiffy

     1 531 400 documents written in 5 minutes

     trunk

     1 314 200 documents written in 5 minutes


Indexing Benoît's warlogs database (https://warlogs.upondata.com/warlogs):

$ time curl http://localhost:5984/warlogs/_design/warlogs/_view/by_date?limit=1
{"total_rows":391832,"offset":0,"rows":[
{"id":"0104D7FC-0219-4C16-8531-97C60A59C70C","key":["2004-01","0104D7FC-0219-4C16-8531-97C60A59C70C"],"value":null}
]}

real	34m39.351s
user	0m0.028s
sys	0m0.036s

Takes roughly the same as trunk, variations of a few minutes (1, 2, 3)
are normal for this view (in machine at least, and always after
clearing the caches with "echo 3 > /proc/sys/vm/drop_caches")


[1] - https://github.com/fdmanana/basho_bench_couch



-- 
Filipe David Manana,
fdmanana@gmail.com, fdmanana@apache.org

"Reasonable men adapt themselves to the world.
 Unreasonable men adapt the world to themselves.
 That's why all progress depends on unreasonable men."

Mime
View raw message