Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
Date: Sat, 2 Apr 2011 15:06:05 +0000 (UTC)
From: "Filipe Manana (JIRA)" <jira@apache.org>
To: dev@couchdb.apache.org
Message-ID: 
 <545314120.30605.1301756765919.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Created] (COUCHDB-1118) Adding a NIF based JSON
 decoding/encoding module
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

Adding a NIF based JSON decoding/encoding module
------------------------------------------------

                 Key: COUCHDB-1118
                 URL: https://issues.apache.org/jira/browse/COUCHDB-1118
             Project: CouchDB
          Issue Type: Improvement
          Components: Database Core
            Reporter: Filipe Manana
             Fix For: 1.2


Currently, all the Erlang based JSON encoders and decoders are very slow, a=
nd decoding and encoding JSON is something that we do basically everywhere.

Via IRC, it recently discussed about adding a JSON NIF encoder/decoder. Dam=
ien also started a thread at the development mailing list about adding NIFs=
 to trunk.

The patch/branch at [1] adds such a JSON encoder/decoder. It is based on Pa=
ul Davis' eep0018 project [2]. Damien made some modifications [3] to it mos=
tly to add support for big numbers (Paul's eep0018 limits the precision to =
32/64 bits) and a few optimizations. I made a few corrections and minor enh=
ancements on top of Damien's fork as well [4]. Finally Beno=C3=AEt identifi=
ed some missing capabilities compared to mochijson2 (on encoding, allow ato=
ms as strings and strings as object properties).

Also, the version added in the patch at [1] uses mochijson2 when the C NIF =
is not loaded. Autotools configuration was adapted to compile the NIF only =
when we're using an OTP release >=3D R13B04 (R13B03 NIF API is too limited =
and suffered many changes compared to R13B04 and R14) - therefore it should=
 work on any OTP release > R13B at least.

I successfully tested this on R13B03, R13B04 and R14B02 in an Ubuntu enviro=
nment.
I'm not sure if it builds at all on Windows - would appreciate if someone c=
ould verify it.
Also, I'm far from being good with the autotools, so I probably missed some=
thing important or I'm doing something in a not very standard way.

This NIF encoder/decoder is about one order of magnitude faster compared to=
 mochijson2 and other Erlang-only solutions such as jsx. A read and writes =
test with relaximation shows this has a very positive impact, specially on =
reads (the EJSON encoding is more expensive than JSON decoding) - http://gr=
aphs.mikeal.couchone.com/#/graph/698bf36b6c64dbd19aa2bef634052381

@Paul, since this is based on your eep0018 effort, do you think any other m=
issing files should be added (README, etap tests, etc)? Also, should we put=
 somewhere a note this is based on your project?

[1] - https://github.com/fdmanana/couchdb/compare/json_nif
[2] - https://github.com/davisp/eep0018
[3] - https://github.com/Damienkatz/eep0018/commits/master
[4] - https://github.com/fdmanana/eep0018/commits/final_damien

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira