couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benoit Chesneau <bchesn...@gmail.com>
Subject Re: svn commit: r1088941 [1/3] - in /couchdb/trunk: ./ src/ src/couchdb/ src/ejson/ src/ejson/yajl/ test/etap/ utils/
Date Tue, 05 Apr 2011 10:00:44 GMT
forgot to say thanks for the work done on eep0018 btw :)

- benoit

On Tue, Apr 5, 2011 at 11:59 AM, Benoit Chesneau <bchesneau@gmail.com> wrote:
> mmm why mochijson is in ejson folder ? Why don't we do the abstraction
> at couchdb level ? More than a cosmectic reason, I think It would be
> better to have a clean separation between codes.
>
> In this case to let couchdb do the abstraction (and reuse the
> couch_util functions we had) then we don't have to duplicate code and
> eventually we could replace ejson by what we want in the future. Also
> it make it easy to use ejson as an external module in apps like
> bigcouch or rcouch or whatever based on CouchDB core.
>
> - benoît
>
> On Tue, Apr 5, 2011 at 11:42 AM,  <fdmanana@apache.org> wrote:
>> Author: fdmanana
>> Date: Tue Apr  5 09:42:41 2011
>> New Revision: 1088941
>>
>> URL: http://svn.apache.org/viewvc?rev=1088941&view=rev
>> Log:
>> Added ejson application
>>
>> This is a NIF based JSON decoder/encoder based on Paul Davis' eep0018
>> implementation (https://github.com/davisp/eep0018/), with some modifications
>> from Damien (big number support and optimizations) on top, plus a few fixes
>> from my side and Benoît on top of Damien's fork.
>> This module fallbacks to mochijson2 when the NIF is not loaded or compiled.
>> The NIF is only compiled and used if we're using an OTP release >= R13B04.
>>
>> Thanks everyone. Closes COUCHDB-1118.
>>
>>
>> Added:
>>    couchdb/trunk/src/ejson/
>>    couchdb/trunk/src/ejson/Makefile.am
>>    couchdb/trunk/src/ejson/decode.c
>>    couchdb/trunk/src/ejson/ejson.app.in
>>    couchdb/trunk/src/ejson/ejson.c
>>    couchdb/trunk/src/ejson/ejson.erl
>>    couchdb/trunk/src/ejson/encode.c
>>    couchdb/trunk/src/ejson/erl_nif_compat.h
>>    couchdb/trunk/src/ejson/mochijson2.erl
>>    couchdb/trunk/src/ejson/mochinum.erl
>>    couchdb/trunk/src/ejson/yajl/
>>    couchdb/trunk/src/ejson/yajl/yajl.c
>>    couchdb/trunk/src/ejson/yajl/yajl_alloc.c
>>    couchdb/trunk/src/ejson/yajl/yajl_alloc.h
>>    couchdb/trunk/src/ejson/yajl/yajl_buf.c
>>    couchdb/trunk/src/ejson/yajl/yajl_buf.h
>>    couchdb/trunk/src/ejson/yajl/yajl_bytestack.h
>>    couchdb/trunk/src/ejson/yajl/yajl_common.h
>>    couchdb/trunk/src/ejson/yajl/yajl_encode.c
>>    couchdb/trunk/src/ejson/yajl/yajl_encode.h
>>    couchdb/trunk/src/ejson/yajl/yajl_gen.c
>>    couchdb/trunk/src/ejson/yajl/yajl_gen.h
>>    couchdb/trunk/src/ejson/yajl/yajl_lex.c
>>    couchdb/trunk/src/ejson/yajl/yajl_lex.h
>>    couchdb/trunk/src/ejson/yajl/yajl_parse.h
>>    couchdb/trunk/src/ejson/yajl/yajl_parser.c
>>    couchdb/trunk/src/ejson/yajl/yajl_parser.h
>> Modified:
>>    couchdb/trunk/.gitignore
>>    couchdb/trunk/NOTICE
>>    couchdb/trunk/configure.ac
>>    couchdb/trunk/src/Makefile.am
>>    couchdb/trunk/src/couchdb/couch_api_wrap.erl
>>    couchdb/trunk/src/couchdb/couch_db.hrl
>>    couchdb/trunk/src/couchdb/couch_httpd_external.erl
>>    couchdb/trunk/src/couchdb/couch_os_process.erl
>>    couchdb/trunk/src/couchdb/couch_util.erl
>>    couchdb/trunk/test/etap/130-attachments-md5.t
>>    couchdb/trunk/test/etap/140-attachment-comp.t
>>    couchdb/trunk/test/etap/150-invalid-view-seq.t
>>    couchdb/trunk/test/etap/160-vhosts.t
>>    couchdb/trunk/test/etap/171-os-daemons-config.es
>>    couchdb/trunk/test/etap/173-os-daemon-cfg-register.es
>>    couchdb/trunk/test/etap/test_util.erl.in
>>    couchdb/trunk/utils/Makefile.am
>>
>> Modified: couchdb/trunk/.gitignore
>> URL: http://svn.apache.org/viewvc/couchdb/trunk/.gitignore?rev=1088941&r1=1088940&r2=1088941&view=diff
>> ==============================================================================
>> --- couchdb/trunk/.gitignore (original)
>> +++ couchdb/trunk/.gitignore Tue Apr  5 09:42:41 2011
>> @@ -64,6 +64,10 @@ src/couchdb/priv/couchspawnkillable
>>  src/couchdb/priv/stat_descriptions.cfg
>>  src/erlang-oauth/oauth.app
>>  src/ibrowse/ibrowse.app
>> +src/ejson/ejson.app
>> +src/ejson/.deps/
>> +src/ejson/.libs/
>> +src/ejson/priv
>>  src/mochiweb/mochiweb.app
>>  test/local.ini
>>  test/etap/run
>>
>> Modified: couchdb/trunk/NOTICE
>> URL: http://svn.apache.org/viewvc/couchdb/trunk/NOTICE?rev=1088941&r1=1088940&r2=1088941&view=diff
>> ==============================================================================
>> --- couchdb/trunk/NOTICE (original)
>> +++ couchdb/trunk/NOTICE Tue Apr  5 09:42:41 2011
>> @@ -49,3 +49,13 @@ This product also includes the following
>>  * jspec.js (http://visionmedia.github.com/jspec/)
>>
>>   Copyright 2010 TJ Holowaychuk <tj@vision-media.ca>
>> +
>> + * yajl (http://lloyd.github.com/yajl/)
>> +
>> +  Copyright 2010, Lloyd Hilaiel
>> +
>> + * ejson
>> +
>> +  Based on Paul Davis' eep0018 implementation (https://github.com/davisp/eep0018/),
>> +  with some modifications from Damien Katz, Filipe Manana and Benoît Chesneau.
>> +  This application uses yajl.
>> \ No newline at end of file
>>
>> Modified: couchdb/trunk/configure.ac
>> URL: http://svn.apache.org/viewvc/couchdb/trunk/configure.ac?rev=1088941&r1=1088940&r2=1088941&view=diff
>> ==============================================================================
>> --- couchdb/trunk/configure.ac (original)
>> +++ couchdb/trunk/configure.ac Tue Apr  5 09:42:41 2011
>> @@ -262,6 +262,10 @@ if test `echo $version | ${AWK} "{print
>>     fi
>>  fi
>>
>> +otp_release="`${ERL} -noshell -eval 'io:put_chars(erlang:system_info(otp_release)).' -s erlang halt`"
>> +AC_SUBST(otp_release)
>> +AM_CONDITIONAL([USE_OTP_NIFS], [test x$otp_release \> xR13B03])
>> +
>>  has_crypto=`${ERL} -eval "case application:load(crypto) of ok -> ok; _ -> exit(no_crypto) end." -noshell -s init stop`
>>
>>  if test -n "$has_crypto"; then
>> @@ -419,6 +423,7 @@ AC_CONFIG_FILES([src/erlang-oauth/Makefi
>>  AC_CONFIG_FILES([src/etap/Makefile])
>>  AC_CONFIG_FILES([src/ibrowse/Makefile])
>>  AC_CONFIG_FILES([src/mochiweb/Makefile])
>> +AC_CONFIG_FILES([src/ejson/Makefile])
>>  AC_CONFIG_FILES([test/Makefile])
>>  AC_CONFIG_FILES([test/bench/Makefile])
>>  AC_CONFIG_FILES([test/etap/Makefile])
>>
>> Modified: couchdb/trunk/src/Makefile.am
>> URL: http://svn.apache.org/viewvc/couchdb/trunk/src/Makefile.am?rev=1088941&r1=1088940&r2=1088941&view=diff
>> ==============================================================================
>> --- couchdb/trunk/src/Makefile.am (original)
>> +++ couchdb/trunk/src/Makefile.am Tue Apr  5 09:42:41 2011
>> @@ -10,4 +10,4 @@
>>  ## License for the specific language governing permissions and limitations under
>>  ## the License.
>>
>> -SUBDIRS = couchdb erlang-oauth etap ibrowse mochiweb
>> +SUBDIRS = couchdb erlang-oauth etap ibrowse mochiweb ejson
>>
>> Modified: couchdb/trunk/src/couchdb/couch_api_wrap.erl
>> URL: http://svn.apache.org/viewvc/couchdb/trunk/src/couchdb/couch_api_wrap.erl?rev=1088941&r1=1088940&r2=1088941&view=diff
>> ==============================================================================
>> --- couchdb/trunk/src/couchdb/couch_api_wrap.erl (original)
>> +++ couchdb/trunk/src/couchdb/couch_api_wrap.erl Tue Apr  5 09:42:41 2011
>> @@ -427,7 +427,7 @@ options_to_query_args([revs | Rest], Acc
>>  options_to_query_args([{open_revs, all} | Rest], Acc) ->
>>     options_to_query_args(Rest, [{"open_revs", "all"} | Acc]);
>>  options_to_query_args([{open_revs, Revs} | Rest], Acc) ->
>> -    JsonRevs = ?JSON_ENCODE(couch_doc:revs_to_strs(Revs)),
>> +    JsonRevs = ?b2l(?JSON_ENCODE(couch_doc:revs_to_strs(Revs))),
>>     options_to_query_args(Rest, [{"open_revs", JsonRevs} | Acc]).
>>
>>
>>
>> Modified: couchdb/trunk/src/couchdb/couch_db.hrl
>> URL: http://svn.apache.org/viewvc/couchdb/trunk/src/couchdb/couch_db.hrl?rev=1088941&r1=1088940&r2=1088941&view=diff
>> ==============================================================================
>> --- couchdb/trunk/src/couchdb/couch_db.hrl (original)
>> +++ couchdb/trunk/src/couchdb/couch_db.hrl Tue Apr  5 09:42:41 2011
>> @@ -20,8 +20,8 @@
>>  % the lowest possible database sequence number
>>  -define(LOWEST_SEQ, 0).
>>
>> --define(JSON_ENCODE(V), couch_util:json_encode(V)).
>> --define(JSON_DECODE(V), couch_util:json_decode(V)).
>> +-define(JSON_ENCODE(V), ejson:encode(V)).
>> +-define(JSON_DECODE(V), ejson:decode(V)).
>>
>>  -define(b2l(V), binary_to_list(V)).
>>  -define(l2b(V), list_to_binary(V)).
>>
>> Modified: couchdb/trunk/src/couchdb/couch_httpd_external.erl
>> URL: http://svn.apache.org/viewvc/couchdb/trunk/src/couchdb/couch_httpd_external.erl?rev=1088941&r1=1088940&r2=1088941&view=diff
>> ==============================================================================
>> --- couchdb/trunk/src/couchdb/couch_httpd_external.erl (original)
>> +++ couchdb/trunk/src/couchdb/couch_httpd_external.erl Tue Apr  5 09:42:41 2011
>> @@ -105,11 +105,11 @@ json_query_keys({Json}) ->
>>  json_query_keys([], Acc) ->
>>     {lists:reverse(Acc)};
>>  json_query_keys([{<<"startkey">>, Value} | Rest], Acc) ->
>> -    json_query_keys(Rest, [{<<"startkey">>, couch_util:json_decode(Value)}|Acc]);
>> +    json_query_keys(Rest, [{<<"startkey">>, ?JSON_DECODE(Value)}|Acc]);
>>  json_query_keys([{<<"endkey">>, Value} | Rest], Acc) ->
>> -    json_query_keys(Rest, [{<<"endkey">>, couch_util:json_decode(Value)}|Acc]);
>> +    json_query_keys(Rest, [{<<"endkey">>, ?JSON_DECODE(Value)}|Acc]);
>>  json_query_keys([{<<"key">>, Value} | Rest], Acc) ->
>> -    json_query_keys(Rest, [{<<"key">>, couch_util:json_decode(Value)}|Acc]);
>> +    json_query_keys(Rest, [{<<"key">>, ?JSON_DECODE(Value)}|Acc]);
>>  json_query_keys([Term | Rest], Acc) ->
>>     json_query_keys(Rest, [Term|Acc]).
>>
>>
>> Modified: couchdb/trunk/src/couchdb/couch_os_process.erl
>> URL: http://svn.apache.org/viewvc/couchdb/trunk/src/couchdb/couch_os_process.erl?rev=1088941&r1=1088940&r2=1088941&view=diff
>> ==============================================================================
>> --- couchdb/trunk/src/couchdb/couch_os_process.erl (original)
>> +++ couchdb/trunk/src/couchdb/couch_os_process.erl Tue Apr  5 09:42:41 2011
>> @@ -60,7 +60,7 @@ prompt(Pid, Data) ->
>>  % Utility functions for reading and writing
>>  % in custom functions
>>  writeline(OsProc, Data) when is_record(OsProc, os_proc) ->
>> -    port_command(OsProc#os_proc.port, Data ++ "\n").
>> +    port_command(OsProc#os_proc.port, [Data, $\n]).
>>
>>  readline(#os_proc{} = OsProc) ->
>>     readline(OsProc, []).
>>
>> Modified: couchdb/trunk/src/couchdb/couch_util.erl
>> URL: http://svn.apache.org/viewvc/couchdb/trunk/src/couchdb/couch_util.erl?rev=1088941&r1=1088940&r2=1088941&view=diff
>> ==============================================================================
>> --- couchdb/trunk/src/couchdb/couch_util.erl (original)
>> +++ couchdb/trunk/src/couchdb/couch_util.erl Tue Apr  5 09:42:41 2011
>> @@ -21,7 +21,6 @@
>>  -export([get_nested_json_value/2, json_user_ctx/1]).
>>  -export([proplist_apply_field/2, json_apply_field/2]).
>>  -export([to_binary/1, to_integer/1, to_list/1, url_encode/1]).
>> --export([json_encode/1, json_decode/1]).
>>  -export([verify/2,simple_call/2,shutdown_sync/1]).
>>  -export([get_value/2, get_value/3]).
>>  -export([md5/1, md5_init/0, md5_update/2, md5_final/1]).
>> @@ -374,22 +373,6 @@ url_encode([H|T]) ->
>>  url_encode([]) ->
>>     [].
>>
>> -json_encode(V) ->
>> -    Handler =
>> -    fun({L}) when is_list(L) ->
>> -        {struct,L};
>> -    (Bad) ->
>> -        exit({json_encode, {bad_term, Bad}})
>> -    end,
>> -    (mochijson2:encoder([{handler, Handler}]))(V).
>> -
>> -json_decode(V) ->
>> -    try (mochijson2:decoder([{object_hook, fun({struct,L}) -> {L} end}]))(V)
>> -    catch
>> -        _Type:_Error ->
>> -            throw({invalid_json,V})
>> -    end.
>> -
>>  verify([X|RestX], [Y|RestY], Result) ->
>>     verify(RestX, RestY, (X bxor Y) bor Result);
>>  verify([], [], Result) ->
>>
>> Added: couchdb/trunk/src/ejson/Makefile.am
>> URL: http://svn.apache.org/viewvc/couchdb/trunk/src/ejson/Makefile.am?rev=1088941&view=auto
>> ==============================================================================
>> --- couchdb/trunk/src/ejson/Makefile.am (added)
>> +++ couchdb/trunk/src/ejson/Makefile.am Tue Apr  5 09:42:41 2011
>> @@ -0,0 +1,86 @@
>> +## Licensed under the Apache License, Version 2.0 (the "License"); you may not
>> +## use this file except in compliance with the License. You may obtain a copy of
>> +## the License at
>> +##
>> +##   http://www.apache.org/licenses/LICENSE-2.0
>> +##
>> +## Unless required by applicable law or agreed to in writing, software
>> +## distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
>> +## WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
>> +## License for the specific language governing permissions and limitations under
>> +## the License.
>> +
>> +ejsonebindir = $(localerlanglibdir)/ejson-0.1.0/ebin
>> +ejsonprivdir = $(localerlanglibdir)/ejson-0.1.0/priv
>> +
>> +CLEANFILES = \
>> +    $(ejsonebin_make_generated_file_list) \
>> +    $(ejsonpriv_make_generated_file_list)
>> +
>> +if USE_OTP_NIFS
>> +ejsonpriv_LTLIBRARIES = ejson.la
>> +endif
>> +
>> +EJSON_C_SRCS = \
>> +       ejson.c \
>> +       decode.c \
>> +       encode.c \
>> +       yajl/yajl_alloc.c \
>> +       yajl/yajl_buf.c \
>> +       yajl/yajl.c \
>> +       yajl/yajl_encode.c \
>> +       yajl/yajl_gen.c \
>> +       yajl/yajl_lex.c \
>> +       yajl/yajl_parser.c
>> +
>> +if USE_OTP_NIFS
>> +ejson_la_SOURCES = $(EJSON_C_SRCS)
>> +ejson_la_LDFLAGS = -module -avoid-version
>> +
>> +if WINDOWS
>> +ejson_la_LDFLAGS += -no-undefined
>> +endif
>> +endif
>> +
>> +ejson_file_collection = \
>> +    ejson.app.in \
>> +    ejson.erl \
>> +    mochijson2.erl \
>> +    mochinum.erl \
>> +    $(JSON_C_SRCS)
>> +
>> +ejsonebin_make_generated_file_list = \
>> +    ejson.app \
>> +    ejson.beam \
>> +    mochijson2.beam \
>> +    mochinum.beam
>> +
>> +ejsonebin_DATA = \
>> +    $(ejsonebin_make_generated_file_list)
>> +
>> +EXTRA_DIST =  \
>> +    $(ejson_file_collection) \
>> +    erl_nif_compat.h \
>> +    yajl/yajl_alloc.h \
>> +    yajl/yajl_buf.h \
>> +    yajl/yajl_bytestack.h \
>> +    yajl/yajl_common.h \
>> +    yajl/yajl_encode.h \
>> +    yajl/yajl_gen.h \
>> +    yajl/yajl_lex.h \
>> +    yajl/yajl_parse.h \
>> +    yajl/yajl_parser.h \
>> +    priv
>> +
>> +if USE_OTP_NIFS
>> +priv/ejson.so: .libs/ejson.so
>> +       $(LN_S) .libs priv
>> +
>> +all: priv/ejson.so
>> +endif
>> +
>> +%.app: %.app.in
>> +       cp $< $@
>> +
>> +%.beam: %.erl
>> +       $(ERLC) $(ERLC_FLAGS) $<
>>
>> Added: couchdb/trunk/src/ejson/decode.c
>> URL: http://svn.apache.org/viewvc/couchdb/trunk/src/ejson/decode.c?rev=1088941&view=auto
>> ==============================================================================
>> --- couchdb/trunk/src/ejson/decode.c (added)
>> +++ couchdb/trunk/src/ejson/decode.c Tue Apr  5 09:42:41 2011
>> @@ -0,0 +1,296 @@
>> +#include <assert.h>
>> +#include <stdio.h>
>> +#include <string.h>
>> +
>> +#include "erl_nif.h"
>> +#include "erl_nif_compat.h"
>> +#include "yajl/yajl_parse.h"
>> +#include "yajl/yajl_parser.h"
>> +#include "yajl/yajl_lex.h"
>> +
>> +typedef struct {
>> +    ERL_NIF_TERM head;
>> +    ErlNifEnv* env;
>> +} decode_ctx;
>> +
>> +#define ENV(ctxarg) (((decode_ctx*)ctxarg)->env)
>> +
>> +#define CONTINUE 1
>> +#define CANCEL 0
>> +
>> +
>> +static ERL_NIF_TERM
>> +make_error(yajl_handle handle, ErlNifEnv* env)
>> +{
>> +    char* yajlError = (char*) yajl_get_error(handle, 0, NULL, 0);
>> +    ERL_NIF_TERM errMsg;
>> +
>> +    if(yajlError != NULL)
>> +    {
>> +        errMsg = enif_make_string(env, yajlError, ERL_NIF_LATIN1);
>> +        yajl_free_error(handle, (unsigned char*) yajlError);
>> +    }
>> +    else
>> +    {
>> +        errMsg = enif_make_string(env, "unknown parse error", ERL_NIF_LATIN1);
>> +    }
>> +
>> +    return enif_make_tuple(env, 2,
>> +        enif_make_atom(env, "error"),
>> +        enif_make_tuple(env, 2,
>> +            enif_make_uint(env, handle->bytesConsumed),
>> +            errMsg
>> +        )
>> +    );
>> +}
>> +
>> +
>> +static void
>> +add_to_head(void* vctx, ERL_NIF_TERM newhead)
>> +{
>> +    decode_ctx* ctx = (decode_ctx*)vctx;
>> +    ctx->head = enif_make_list_cell(ctx->env, newhead, ctx->head);
>> +}
>> +
>> +static int
>> +decode_null(void* ctx)
>> +{
>> +    add_to_head(ctx, enif_make_atom(ENV(ctx), "null"));
>> +    return CONTINUE;
>> +}
>> +
>> +static int
>> +decode_boolean(void* ctx, int val)
>> +{
>> +    add_to_head(ctx, enif_make_atom(ENV(ctx), val ? "true" : "false"));
>> +    return CONTINUE;
>> +}
>> +
>> +static int
>> +decode_number(void * ctx, const char * numberVal, unsigned int numberLen)
>> +{
>> +    // scan in the input to see if it's a float or int
>> +
>> +    int numberType = 0; // 0 means integer, 1 means float
>> +    unsigned int i;
>> +    ErlNifBinary bin;
>> +    int missingDot = 1;
>> +    unsigned int expPos;
>> +
>> +    for(i=0; i<numberLen; i++) {
>> +        switch (numberVal[i]) {
>> +        case '.':
>> +            missingDot = 0;
>> +            numberType = 1; // it's  a float
>> +            goto loopend;
>> +        case 'E':
>> +        case 'e':
>> +            expPos = i;
>> +            numberType = 1; // it's  a float
>> +            goto loopend;
>> +        }
>> +    }
>> +loopend:
>> +    if ((numberType == 1) && missingDot)
>> +    {
>> +        if(!enif_alloc_binary_compat(ENV(ctx), numberLen + 2, &bin))
>> +        {
>> +            return CANCEL;
>> +        }
>> +        memcpy(bin.data, numberVal, expPos);
>> +        bin.data[expPos] = '.';
>> +        bin.data[expPos + 1] = '0';
>> +        memcpy(bin.data + expPos + 2, numberVal + expPos, numberLen - expPos);
>> +    }
>> +    else
>> +    {
>> +        if(!enif_alloc_binary_compat(ENV(ctx), numberLen, &bin))
>> +        {
>> +            return CANCEL;
>> +        }
>> +        memcpy(bin.data, numberVal, numberLen);
>> +    }
>> +    add_to_head(ctx, enif_make_tuple(ENV(ctx), 2,
>> +                        enif_make_int(ENV(ctx), numberType),
>> +                        enif_make_binary(ENV(ctx), &bin)));
>> +    return CONTINUE;
>> +}
>> +
>> +
>> +
>> +static int
>> +decode_string(void* ctx, const unsigned char* data, unsigned int size)
>> +{
>> +    ErlNifBinary bin;
>> +    if(!enif_alloc_binary_compat(ENV(ctx), size, &bin))
>> +    {
>> +        return CANCEL;
>> +    }
>> +    memcpy(bin.data, data, size);
>> +    add_to_head(ctx, enif_make_binary(ENV(ctx), &bin));
>> +    return CONTINUE;
>> +}
>> +
>> +static int
>> +decode_start_array(void* ctx)
>> +{
>> +    add_to_head(ctx, enif_make_int(ENV(ctx), 0));
>> +    return CONTINUE;
>> +}
>> +
>> +
>> +static int
>> +decode_end_array(void* ctx)
>> +{
>> +    add_to_head(ctx, enif_make_int(ENV(ctx), 1));
>> +    return CONTINUE;
>> +}
>> +
>> +
>> +static int
>> +decode_start_map(void* ctx)
>> +{
>> +    add_to_head(ctx, enif_make_int(ENV(ctx), 2));
>> +    return CONTINUE;
>> +}
>> +
>> +
>> +static int
>> +decode_end_map(void* ctx)
>> +{
>> +    add_to_head(ctx, enif_make_int(ENV(ctx), 3));
>> +    return CONTINUE;
>> +}
>> +
>> +
>> +static int
>> +decode_map_key(void* ctx, const unsigned char* data, unsigned int size)
>> +{
>> +    ErlNifBinary bin;
>> +    if(!enif_alloc_binary_compat(ENV(ctx), size, &bin))
>> +    {
>> +       return CANCEL;
>> +    }
>> +    memcpy(bin.data, data, size);
>> +    add_to_head(ctx, enif_make_tuple(ENV(ctx), 2,
>> +                        enif_make_int(ENV(ctx), 3),
>> +                        enif_make_binary(ENV(ctx), &bin)));
>> +    return CONTINUE;
>> +}
>> +
>> +static yajl_callbacks
>> +decoder_callbacks = {
>> +    decode_null,
>> +    decode_boolean,
>> +    NULL,
>> +    NULL,
>> +    decode_number,
>> +    decode_string,
>> +    decode_start_map,
>> +    decode_map_key,
>> +    decode_end_map,
>> +    decode_start_array,
>> +    decode_end_array
>> +};
>> +
>> +static int
>> +check_rest(unsigned char* data, unsigned int size, unsigned int used)
>> +{
>> +    unsigned int i = 0;
>> +    for(i = used; i < size; i++)
>> +    {
>> +        switch(data[i])
>> +        {
>> +            case ' ':
>> +            case '\t':
>> +            case '\r':
>> +            case '\n':
>> +                continue;
>> +            default:
>> +                return CANCEL;
>> +        }
>> +    }
>> +
>> +    return CONTINUE;
>> +}
>> +
>> +ERL_NIF_TERM
>> +reverse_tokens(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[])
>> +{
>> +    decode_ctx ctx;
>> +    yajl_parser_config conf = {0, 1}; // No comments, check utf8
>> +    yajl_handle handle = yajl_alloc(&decoder_callbacks, &conf, NULL, &ctx);
>> +    yajl_status status;
>> +    unsigned int used;
>> +    ErlNifBinary bin;
>> +    ERL_NIF_TERM ret;
>> +
>> +    ctx.env = env;
>> +    ctx.head = enif_make_list_from_array(env, NULL, 0);
>> +
>> +    if(!enif_inspect_iolist_as_binary(env, argv[0], &bin))
>> +    {
>> +        ret = enif_make_badarg(env);
>> +        goto done;
>> +    }
>> +
>> +    status = yajl_parse(handle, bin.data, bin.size);
>> +    used = handle->bytesConsumed;
>> +
>> +    // Parsing something like "2.0" (without quotes) will
>> +    // cause a spurious semi-error. We add the extra size
>> +    // check so that "2008-20-10" doesn't pass.
>> +    if(status == yajl_status_insufficient_data && used == bin.size)
>> +    {
>> +        status = yajl_parse_complete(handle);
>> +    }
>> +
>> +    if(status == yajl_status_ok && used != bin.size)
>> +    {
>> +        if(check_rest(bin.data, bin.size, used) == CANCEL)
>> +        {
>> +            ret = enif_make_tuple(env, 2,
>> +                enif_make_atom(env, "error"),
>> +                enif_make_atom(env, "garbage_after_value")
>> +            );
>> +            goto done;
>> +        }
>> +    }
>> +
>> +    switch(status)
>> +    {
>> +        case yajl_status_ok:
>> +            ret = enif_make_tuple(env, 2, enif_make_atom(env, "ok"), ctx.head);
>> +            goto done;
>> +
>> +        case yajl_status_error:
>> +            ret = make_error(handle, env);
>> +            goto done;
>> +
>> +        case yajl_status_insufficient_data:
>> +            ret = enif_make_tuple(env, 2,
>> +                enif_make_atom(env, "error"),
>> +                enif_make_atom(env, "insufficient_data")
>> +            );
>> +            goto done;
>> +
>> +        case yajl_status_client_canceled:
>> +        /* the only time we do this is when we can't allocate a binary. */
>> +            ret = enif_make_tuple(env, 2,
>> +                enif_make_atom(env, "error"),
>> +                enif_make_atom(env, "insufficient_memory")
>> +            );
>> +            goto done;
>> +
>> +        default:
>> +            ret = enif_make_tuple(env, 2,
>> +                enif_make_atom(env, "error"),
>> +                enif_make_atom(env, "unknown")
>> +            );
>> +            goto done;
>> +    }
>> +
>> +done:
>> +    if(handle != NULL) yajl_free(handle);
>> +    return ret;
>> +}
>>
>> Added: couchdb/trunk/src/ejson/ejson.app.in
>> URL: http://svn.apache.org/viewvc/couchdb/trunk/src/ejson/ejson.app.in?rev=1088941&view=auto
>> ==============================================================================
>> --- couchdb/trunk/src/ejson/ejson.app.in (added)
>> +++ couchdb/trunk/src/ejson/ejson.app.in Tue Apr  5 09:42:41 2011
>> @@ -0,0 +1,9 @@
>> +{application, ejson, [
>> +    {description, "EJSON - decode and encode JSON into/from Erlang terms"},
>> +    {vsn, "0.1.0"},
>> +    {modules, [ejson]},
>> +    {registered, []},
>> +    {applications, [kernel, stdlib]},
>> +    {env, []}
>> +]}.
>> +
>>
>> Added: couchdb/trunk/src/ejson/ejson.c
>> URL: http://svn.apache.org/viewvc/couchdb/trunk/src/ejson/ejson.c?rev=1088941&view=auto
>> ==============================================================================
>> --- couchdb/trunk/src/ejson/ejson.c (added)
>> +++ couchdb/trunk/src/ejson/ejson.c Tue Apr  5 09:42:41 2011
>> @@ -0,0 +1,30 @@
>> +#include "erl_nif.h"
>> +
>> +ERL_NIF_TERM final_encode(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]);
>> +ERL_NIF_TERM reverse_tokens(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]);
>> +
>> +int
>> +on_load(ErlNifEnv* env, void** priv_data, ERL_NIF_TERM info)
>> +{
>> +    return 0;
>> +}
>> +
>> +int
>> +on_reload(ErlNifEnv* env, void** priv_data, ERL_NIF_TERM info)
>> +{
>> +    return 0;
>> +}
>> +
>> +int
>> +on_upgrade(ErlNifEnv* env, void** priv_data, void** old_data, ERL_NIF_TERM info)
>> +{
>> +    return 0;
>> +}
>> +
>> +static ErlNifFunc nif_funcs[] =
>> +{
>> +    {"final_encode", 1, final_encode},
>> +    {"reverse_tokens", 1, reverse_tokens}
>> +};
>> +
>> +ERL_NIF_INIT(ejson, nif_funcs, &on_load, &on_reload, &on_upgrade, NULL);
>>
>> Added: couchdb/trunk/src/ejson/ejson.erl
>> URL: http://svn.apache.org/viewvc/couchdb/trunk/src/ejson/ejson.erl?rev=1088941&view=auto
>> ==============================================================================
>> --- couchdb/trunk/src/ejson/ejson.erl (added)
>> +++ couchdb/trunk/src/ejson/ejson.erl Tue Apr  5 09:42:41 2011
>> @@ -0,0 +1,151 @@
>> +-module(ejson).
>> +-export([encode/1, decode/1]).
>> +-on_load(init/0).
>> +
>> +init() ->
>> +    SoName = case code:priv_dir(ejson) of
>> +    {error, bad_name} ->
>> +        case filelib:is_dir(filename:join(["..", priv])) of
>> +        true ->
>> +            filename:join(["..", priv, ejson]);
>> +        false ->
>> +            filename:join([priv, ejson])
>> +        end;
>> +    Dir ->
>> +        filename:join(Dir, ejson)
>> +    end,
>> +    (catch erlang:load_nif(SoName, 0)),
>> +    ok.
>> +
>> +
>> +decode(IoList) ->
>> +    try
>> +        nif_decode(IoList)
>> +    catch exit:ejson_nif_not_loaded ->
>> +        erl_decode(IoList)
>> +    end.
>> +
>> +encode(EJson) ->
>> +    try
>> +        nif_encode(EJson)
>> +    catch exit:ejson_nif_not_loaded ->
>> +        erl_encode(EJson)
>> +    end.
>> +
>> +
>> +nif_decode(IoList) ->
>> +    case reverse_tokens(IoList) of
>> +    {ok, ReverseTokens} ->
>> +        [[EJson]] = make_ejson(ReverseTokens, [[]]),
>> +        EJson;
>> +    Error ->
>> +        throw({invalid_json, {Error, IoList}})
>> +    end.
>> +
>> +
>> +erl_decode(IoList) ->
>> +    try
>> +        (mochijson2:decoder([{object_hook, fun({struct, L}) -> {L} end}]))(IoList)
>> +    catch _Type:Error ->
>> +        throw({invalid_json, {Error, IoList}})
>> +    end.
>> +
>> +
>> +nif_encode(EJson) ->
>> +    RevList = encode_rev(EJson),
>> +    final_encode(lists:reverse(lists:flatten([RevList]))).
>> +
>> +
>> +erl_encode(EJson) ->
>> +    Opts = [{handler, fun mochi_encode_handler/1}],
>> +    iolist_to_binary((mochijson2:encoder(Opts))(EJson)).
>> +
>> +mochi_encode_handler({L}) when is_list(L) ->
>> +    {struct, L};
>> +mochi_encode_handler(Bad) ->
>> +    exit({json_encode, {bad_term, Bad}}).
>> +
>> +
>> +% Encode the json into a reverse list that's almost an iolist
>> +% everything in the list is the final output except for tuples with
>> +% {0, Strings} and {1, Floats}, which are to be converted to strings
>> +% inside the NIF.
>> +encode_rev(true) ->
>> +    <<"true">>;
>> +encode_rev(false) ->
>> +    <<"false">>;
>> +encode_rev(null) ->
>> +    <<"null">>;
>> +encode_rev(I) when is_integer(I) ->
>> +    list_to_binary(integer_to_list(I));
>> +encode_rev(S) when is_binary(S) ->
>> +    {0, S};
>> +encode_rev(S) when is_atom(S) ->
>> +    {0, list_to_binary(atom_to_list(S))};
>> +encode_rev(F) when is_float(F) ->
>> +    {1, F};
>> +encode_rev({Props}) when is_list(Props) ->
>> +    encode_proplist_rev(Props, [<<"{">>]);
>> +encode_rev(Array) when is_list(Array) ->
>> +    encode_array_rev(Array, [<<"[">>]);
>> +encode_rev(Bad) ->
>> +    throw({json_encode, {bad_term, Bad}}).
>> +
>> +
>> +encode_array_rev([], Acc) ->
>> +    [<<"]">> | Acc];
>> +encode_array_rev([Val | Rest], [<<"[">>]) ->
>> +    encode_array_rev(Rest, [encode_rev(Val), <<"[">>]);
>> +encode_array_rev([Val | Rest], Acc) ->
>> +    encode_array_rev(Rest, [encode_rev(Val), <<",">> | Acc]).
>> +
>> +
>> +encode_proplist_rev([], Acc) ->
>> +    [<<"}">> | Acc];
>> +encode_proplist_rev([{Key,Val} | Rest], [<<"{">>]) ->
>> +    encode_proplist_rev(
>> +        Rest, [encode_rev(Val), <<":">>, {0, as_binary(Key)}, <<"{">>]);
>> +encode_proplist_rev([{Key,Val} | Rest], Acc) ->
>> +    encode_proplist_rev(
>> +        Rest, [encode_rev(Val), <<":">>, {0, as_binary(Key)}, <<",">> | Acc]).
>> +
>> +as_binary(B) when is_binary(B) ->
>> +    B;
>> +as_binary(A) when is_atom(A) ->
>> +    list_to_binary(atom_to_list(A));
>> +as_binary(L) when is_list(L) ->
>> +    list_to_binary(L).
>> +
>> +
>> +make_ejson([], Stack) ->
>> +    Stack;
>> +make_ejson([0 | RevEvs], [ArrayValues, PrevValues | RestStack]) ->
>> +    % 0 ArrayStart
>> +    make_ejson(RevEvs, [[ArrayValues | PrevValues] | RestStack]);
>> +make_ejson([1 | RevEvs], Stack) ->
>> +    % 1 ArrayEnd
>> +    make_ejson(RevEvs, [[] | Stack]);
>> +make_ejson([2 | RevEvs], [ObjValues, PrevValues | RestStack]) ->
>> +    % 2 ObjectStart
>> +    make_ejson(RevEvs, [[{ObjValues} | PrevValues] | RestStack]);
>> +make_ejson([3 | RevEvs], Stack) ->
>> +    % 3 ObjectEnd
>> +    make_ejson(RevEvs, [[] | Stack]);
>> +make_ejson([{0, Value} | RevEvs], [Vals | RestStack] = _Stack) ->
>> +    % {0, IntegerString}
>> +    make_ejson(RevEvs, [[list_to_integer(binary_to_list(Value)) | Vals] | RestStack]);
>> +make_ejson([{1, Value} | RevEvs], [Vals | RestStack] = _Stack) ->
>> +    % {1, FloatString}
>> +    make_ejson(RevEvs, [[list_to_float(binary_to_list(Value)) | Vals] | RestStack]);
>> +make_ejson([{3, String} | RevEvs], [[PrevValue|RestObject] | RestStack] = _Stack) ->
>> +    % {3 , ObjectKey}
>> +    make_ejson(RevEvs, [[{String, PrevValue}|RestObject] | RestStack]);
>> +make_ejson([Value | RevEvs], [Vals | RestStack] = _Stack) ->
>> +    make_ejson(RevEvs, [[Value | Vals] | RestStack]).
>> +
>> +
>> +reverse_tokens(_) ->
>> +    exit(ejson_nif_not_loaded).
>> +
>> +final_encode(_) ->
>> +    exit(ejson_nif_not_loaded).
>>
>> Added: couchdb/trunk/src/ejson/encode.c
>> URL: http://svn.apache.org/viewvc/couchdb/trunk/src/ejson/encode.c?rev=1088941&view=auto
>> ==============================================================================
>> --- couchdb/trunk/src/ejson/encode.c (added)
>> +++ couchdb/trunk/src/ejson/encode.c Tue Apr  5 09:42:41 2011
>> @@ -0,0 +1,164 @@
>> +#include <stdio.h>
>> +#include <string.h>
>> +#include <math.h>
>> +
>> +#include "erl_nif.h"
>> +#include "erl_nif_compat.h"
>> +#include "yajl/yajl_encode.h"
>> +
>> +#define SUCCESS 0
>> +#define NOMEM 1
>> +#define BADARG 2
>> +
>> +
>> +typedef struct {
>> +    ErlNifEnv* env;
>> +    ErlNifBinary bin;
>> +    size_t fill_offset;
>> +    int error;
>> +} encode_ctx;
>> +
>> +
>> +static int
>> +ensure_buffer(void* vctx, unsigned int len) {
>> +    encode_ctx* ctx = (encode_ctx*)vctx;
>> +    if ((ctx->bin.size - ctx->fill_offset) < len) {
>> +        if(!enif_realloc_binary_compat(ctx->env, &(ctx->bin), (ctx->bin.size * 2) + len)) {
>> +            return NOMEM;
>> +        }
>> +    }
>> +    return SUCCESS;
>> +}
>> +
>> +static void
>> +fill_buffer(void* vctx, const char* str, unsigned int len)
>> +{
>> +    encode_ctx* ctx = (encode_ctx*)vctx;
>> +
>> +    if (ctx->error || (ctx->error = ensure_buffer(vctx, len))) {
>> +        return;
>> +    }
>> +    memcpy(ctx->bin.data + ctx->fill_offset, str, len);
>> +    ctx->fill_offset += len;
>> +}
>> +
>> +/* Json encode the string binary into the ctx.bin,
>> +  with surrounding quotes and all */
>> +static int
>> +encode_string(void* vctx, ERL_NIF_TERM binary)
>> +{
>> +    encode_ctx* ctx = (encode_ctx*)vctx;
>> +    ErlNifBinary bin;
>> +
>> +    if(!enif_inspect_binary(ctx->env, binary, &bin)) {
>> +        return NOMEM;
>> +    }
>> +    fill_buffer(ctx, "\"", 1);
>> +    if (ctx->error) {
>> +        return ctx->error;
>> +    }
>> +    yajl_string_encode2(fill_buffer, ctx, bin.data, bin.size);
>> +    fill_buffer(ctx, "\"", 1);
>> +
>> +    return ctx->error;
>> +}
>> +
>> +static ERL_NIF_TERM
>> +no_mem_error(ErlNifEnv* env)
>> +{
>> +    return enif_make_tuple(env, 2,
>> +            enif_make_atom(env, "error"),
>> +            enif_make_atom(env, "insufficient_memory"));
>> +}
>> +
>> +ERL_NIF_TERM
>> +final_encode(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[])
>> +{
>> +    ERL_NIF_TERM head = argv[0];
>> +    ERL_NIF_TERM term;
>> +    double number;
>> +    encode_ctx ctx;
>> +
>> +    ctx.env = env;
>> +    ctx.fill_offset = 0;
>> +    ctx.error = 0;
>> +
>> +    if (!enif_alloc_binary_compat(env, 100, &ctx.bin)) {
>> +            return no_mem_error(env);
>> +    }
>> +
>> +    while(enif_get_list_cell(env, head, &term, &head)) {
>> +        ErlNifBinary termbin;
>> +        const ERL_NIF_TERM* array;
>> +        int arity;
>> +        int code;
>> +
>> +        // We scan the list, looking for things to write into the binary, or
>> +        // encode and then write into the binary. We encode values that are
>> +        // tuples tagged with a type and a value: {Type, Value} where Type
>> +        // is a an Integer and Value is what is to be encoded
>> +
>> +        if (enif_get_tuple(env, term, &arity, &array)) {
>> +            // It's a tuple to encode and copy
>> +            if (arity != 2 || !enif_get_int(env, array[0], &code)) {
>> +                // not arity 2 or the first element isn't an int
>> +                ctx.error = BADARG;
>> +                goto done;
>> +            }
>> +            if (code == 0) {
>> +                // {0, String}
>> +                if (encode_string(&ctx, array[1]) != SUCCESS) {
>> +                    goto done;
>> +                }
>> +            }
>> +            else {
>> +                // {1, Double}
>> +                if(!enif_get_double(env, array[1], &number)) {
>> +                    ctx.error = BADARG;
>> +                    goto done;
>> +                }
>> +                // We can't encode these.
>> +                if (isnan(number) || isinf(number)) {
>> +                    ctx.error = BADARG;
>> +                    goto done;
>> +                }
>> +                if ((ctx.error = ensure_buffer(&ctx, 32)) != SUCCESS) {
>> +                    goto done;
>> +                }
>> +                // write the string into the buffer
>> +                snprintf((char*)ctx.bin.data+ctx.fill_offset, 32,
>> +                        "%.16g", number);
>> +                // increment the length
>> +                ctx.fill_offset += strlen((char*)ctx.bin.data+ctx.fill_offset);
>> +            }
>> +        } else if (enif_inspect_binary(env, term, &termbin)) {
>> +            // this is a regular binary, copy the contents into the buffer
>> +            fill_buffer(&ctx, (char*)termbin.data, termbin.size);
>> +            if (ctx.error) {
>> +                goto done;
>> +            }
>> +        }
>> +        else {
>> +            //not a binary, not a tuple, wtf!
>> +            ctx.error = BADARG;
>> +            goto done;
>> +        }
>> +    }
>> +done:
>> +    if (ctx.error == NOMEM) {
>> +        enif_release_binary_compat(env, &ctx.bin);
>> +        return no_mem_error(env);
>> +    } else if (ctx.error == BADARG) {
>> +        enif_release_binary_compat(env, &ctx.bin);
>> +        return enif_make_badarg(env);
>> +    }
>> +
>> +    // Resize the binary to our exact final size
>> +    if(!enif_realloc_binary_compat(env, &(ctx.bin), ctx.fill_offset)) {
>> +        enif_release_binary_compat(env, &ctx.bin);
>> +        return no_mem_error(env);
>> +    }
>> +    // make the binary term which transfers ownership
>> +    return enif_make_binary(env, &ctx.bin);
>> +}
>> +
>>
>> Added: couchdb/trunk/src/ejson/erl_nif_compat.h
>> URL: http://svn.apache.org/viewvc/couchdb/trunk/src/ejson/erl_nif_compat.h?rev=1088941&view=auto
>> ==============================================================================
>> --- couchdb/trunk/src/ejson/erl_nif_compat.h (added)
>> +++ couchdb/trunk/src/ejson/erl_nif_compat.h Tue Apr  5 09:42:41 2011
>> @@ -0,0 +1,102 @@
>> +#ifndef ERL_NIF_COMPAT_H_
>> +#define ERL_NIF_COMPAT_H_
>> +
>> +#ifdef __cplusplus
>> +extern "C" {
>> +#endif /* __cplusplus */
>> +
>> +#include "erl_nif.h"
>> +
>> +
>> +#if ERL_NIF_MAJOR_VERSION == 0 && ERL_NIF_MINOR_VERSION == 1
>> +#define OTP_R13B03
>> +#elif ERL_NIF_MAJOR_VERSION == 1 && ERL_NIF_MINOR_VERSION == 0
>> +#define OTP_R13B04
>> +#elif ERL_NIF_MAJOR_VERSION == 2 && ERL_NIF_MINOR_VERSION == 0
>> +#define OTP_R14A
>> +#define OTP_R14B
>> +#define OTP_R14B01
>> +#elif ERL_NIF_MAJOR_VERSION == 2 && ERL_NIF_MINOR_VERSION == 1
>> +#define OTP_R14B02
>> +#endif
>> +
>> +
>> +#ifdef OTP_R13B03
>> +
>> +#define enif_open_resource_type_compat enif_open_resource_type
>> +#define enif_alloc_resource_compat enif_alloc_resource
>> +#define enif_release_resource_compat enif_release_resource
>> +#define enif_alloc_binary_compat enif_alloc_binary
>> +#define enif_alloc_compat enif_alloc
>> +#define enif_release_binary_compat enif_release_binary
>> +#define enif_free_compat enif_free
>> +#define enif_get_atom_compat enif_get_atom
>> +#define enif_priv_data_compat enif_get_data
>> +#define enif_make_uint_compat enif_make_ulong
>> +
>> +#define enif_make_string_compat(E, B, Enc) \
>> +    enif_make_string(E, B)
>> +
>> +#endif /* R13B03 */
>> +
>> +
>> +#ifdef OTP_R13B04
>> +
>> +#define enif_open_resource_type_compat enif_open_resource_type
>> +#define enif_alloc_resource_compat enif_alloc_resource
>> +#define enif_release_resource_compat enif_release_resource
>> +#define enif_alloc_binary_compat enif_alloc_binary
>> +#define enif_realloc_binary_compat enif_realloc_binary
>> +#define enif_release_binary_compat enif_release_binary
>> +#define enif_alloc_compat enif_alloc
>> +#define enif_free_compat enif_free
>> +#define enif_get_atom_compat enif_get_atom
>> +#define enif_priv_data_compat enif_priv_data
>> +#define enif_make_string_compat enif_make_string
>> +#define enif_make_uint_compat enif_make_uint
>> +
>> +#endif /* R13B04 */
>> +
>> +
>> +/* OTP R14 and future releases */
>> +#if !defined(OTP_R13B03) && !defined(OTP_R13B04)
>> +
>> +#define enif_open_resource_type_compat(E, N, D, F, T) \
>> +    enif_open_resource_type(E, NULL, N, D, F, T)
>> +
>> +#define enif_alloc_resource_compat(E, T, S) \
>> +    enif_alloc_resource(T, S)
>> +
>> +#define enif_release_resource_compat(E, H) \
>> +    enif_release_resource(H)
>> +
>> +#define enif_alloc_binary_compat(E, S, B) \
>> +    enif_alloc_binary(S, B)
>> +
>> +#define enif_realloc_binary_compat(E, S, B) \
>> +    enif_realloc_binary(S, B)
>> +
>> +#define enif_release_binary_compat(E, B) \
>> +    enif_release_binary(B)
>> +
>> +#define enif_alloc_compat(E, S) \
>> +    enif_alloc(S)
>> +
>> +#define enif_free_compat(E, P) \
>> +    enif_free(P)
>> +
>> +#define enif_get_atom_compat(E, T, B, S) \
>> +    enif_get_atom(E, T, B, S, ERL_NIF_LATIN1)
>> +
>> +#define enif_priv_data_compat enif_priv_data
>> +#define enif_make_string_compat enif_make_string
>> +#define enif_make_uint_compat enif_make_uint
>> +
>> +#endif  /* R14 and future releases */
>> +
>> +
>> +#ifdef __cplusplus
>> +}
>> +#endif /* __cplusplus */
>> +
>> +#endif /* ERL_NIF_COMPAT_H_ */
>>
>> Added: couchdb/trunk/src/ejson/mochijson2.erl
>> URL: http://svn.apache.org/viewvc/couchdb/trunk/src/ejson/mochijson2.erl?rev=1088941&view=auto
>> ==============================================================================
>> --- couchdb/trunk/src/ejson/mochijson2.erl (added)
>> +++ couchdb/trunk/src/ejson/mochijson2.erl Tue Apr  5 09:42:41 2011
>> @@ -0,0 +1,849 @@
>> +%% @author Bob Ippolito <bob@mochimedia.com>
>> +%% @copyright 2007 Mochi Media, Inc.
>> +
>> +%% @doc Yet another JSON (RFC 4627) library for Erlang. mochijson2 works
>> +%%      with binaries as strings, arrays as lists (without an {array, _})
>> +%%      wrapper and it only knows how to decode UTF-8 (and ASCII).
>> +%%
>> +%%      JSON terms are decoded as follows (javascript -> erlang):
>> +%%      <ul>
>> +%%          <li>{"key": "value"} ->
>> +%%              {struct, [{&lt;&lt;"key">>, &lt;&lt;"value">>}]}</li>
>> +%%          <li>["array", 123, 12.34, true, false, null] ->
>> +%%              [&lt;&lt;"array">>, 123, 12.34, true, false, null]
>> +%%          </li>
>> +%%      </ul>
>> +%%      <ul>
>> +%%          <li>Strings in JSON decode to UTF-8 binaries in Erlang</li>
>> +%%          <li>Objects decode to {struct, PropList}</li>
>> +%%          <li>Numbers decode to integer or float</li>
>> +%%          <li>true, false, null decode to their respective terms.</li>
>> +%%      </ul>
>> +%%      The encoder will accept the same format that the decoder will produce,
>> +%%      but will also allow additional cases for leniency:
>> +%%      <ul>
>> +%%          <li>atoms other than true, false, null will be considered UTF-8
>> +%%              strings (even as a proplist key)
>> +%%          </li>
>> +%%          <li>{json, IoList} will insert IoList directly into the output
>> +%%              with no validation
>> +%%          </li>
>> +%%          <li>{array, Array} will be encoded as Array
>> +%%              (legacy mochijson style)
>> +%%          </li>
>> +%%          <li>A non-empty raw proplist will be encoded as an object as long
>> +%%              as the first pair does not have an atom key of json, struct,
>> +%%              or array
>> +%%          </li>
>> +%%      </ul>
>> +
>> +-module(mochijson2).
>> +-author('bob@mochimedia.com').
>> +-export([encoder/1, encode/1]).
>> +-export([decoder/1, decode/1]).
>> +
>> +% This is a macro to placate syntax highlighters..
>> +-define(Q, $\").
>> +-define(ADV_COL(S, N), S#decoder{offset=N+S#decoder.offset,
>> +                                 column=N+S#decoder.column}).
>> +-define(INC_COL(S), S#decoder{offset=1+S#decoder.offset,
>> +                              column=1+S#decoder.column}).
>> +-define(INC_LINE(S), S#decoder{offset=1+S#decoder.offset,
>> +                               column=1,
>> +                               line=1+S#decoder.line}).
>> +-define(INC_CHAR(S, C),
>> +        case C of
>> +            $\n ->
>> +                S#decoder{column=1,
>> +                          line=1+S#decoder.line,
>> +                          offset=1+S#decoder.offset};
>> +            _ ->
>> +                S#decoder{column=1+S#decoder.column,
>> +                          offset=1+S#decoder.offset}
>> +        end).
>> +-define(IS_WHITESPACE(C),
>> +        (C =:= $\s orelse C =:= $\t orelse C =:= $\r orelse C =:= $\n)).
>> +
>> +%% @type iolist() = [char() | binary() | iolist()]
>> +%% @type iodata() = iolist() | binary()
>> +%% @type json_string() = atom | binary()
>> +%% @type json_number() = integer() | float()
>> +%% @type json_array() = [json_term()]
>> +%% @type json_object() = {struct, [{json_string(), json_term()}]}
>> +%% @type json_iolist() = {json, iolist()}
>> +%% @type json_term() = json_string() | json_number() | json_array() |
>> +%%                     json_object() | json_iolist()
>> +
>> +-record(encoder, {handler=null,
>> +                  utf8=false}).
>> +
>> +-record(decoder, {object_hook=null,
>> +                  offset=0,
>> +                  line=1,
>> +                  column=1,
>> +                  state=null}).
>> +
>> +%% @spec encoder([encoder_option()]) -> function()
>> +%% @doc Create an encoder/1 with the given options.
>> +%% @type encoder_option() = handler_option() | utf8_option()
>> +%% @type utf8_option() = boolean(). Emit unicode as utf8 (default - false)
>> +encoder(Options) ->
>> +    State = parse_encoder_options(Options, #encoder{}),
>> +    fun (O) -> json_encode(O, State) end.
>> +
>> +%% @spec encode(json_term()) -> iolist()
>> +%% @doc Encode the given as JSON to an iolist.
>> +encode(Any) ->
>> +    json_encode(Any, #encoder{}).
>> +
>> +%% @spec decoder([decoder_option()]) -> function()
>> +%% @doc Create a decoder/1 with the given options.
>> +decoder(Options) ->
>> +    State = parse_decoder_options(Options, #decoder{}),
>> +    fun (O) -> json_decode(O, State) end.
>> +
>> +%% @spec decode(iolist()) -> json_term()
>> +%% @doc Decode the given iolist to Erlang terms.
>> +decode(S) ->
>> +    json_decode(S, #decoder{}).
>> +
>> +%% Internal API
>> +
>> +parse_encoder_options([], State) ->
>> +    State;
>> +parse_encoder_options([{handler, Handler} | Rest], State) ->
>> +    parse_encoder_options(Rest, State#encoder{handler=Handler});
>> +parse_encoder_options([{utf8, Switch} | Rest], State) ->
>> +    parse_encoder_options(Rest, State#encoder{utf8=Switch}).
>> +
>> +parse_decoder_options([], State) ->
>> +    State;
>> +parse_decoder_options([{object_hook, Hook} | Rest], State) ->
>> +    parse_decoder_options(Rest, State#decoder{object_hook=Hook}).
>> +
>> +json_encode(true, _State) ->
>> +    <<"true">>;
>> +json_encode(false, _State) ->
>> +    <<"false">>;
>> +json_encode(null, _State) ->
>> +    <<"null">>;
>> +json_encode(I, _State) when is_integer(I) ->
>> +    integer_to_list(I);
>> +json_encode(F, _State) when is_float(F) ->
>> +    mochinum:digits(F);
>> +json_encode(S, State) when is_binary(S); is_atom(S) ->
>> +    json_encode_string(S, State);
>> +json_encode([{K, _}|_] = Props, State) when (K =/= struct andalso
>> +                                             K =/= array andalso
>> +                                             K =/= json) ->
>> +    json_encode_proplist(Props, State);
>> +json_encode({struct, Props}, State) when is_list(Props) ->
>> +    json_encode_proplist(Props, State);
>> +json_encode(Array, State) when is_list(Array) ->
>> +    json_encode_array(Array, State);
>> +json_encode({array, Array}, State) when is_list(Array) ->
>> +    json_encode_array(Array, State);
>> +json_encode({json, IoList}, _State) ->
>> +    IoList;
>> +json_encode(Bad, #encoder{handler=null}) ->
>> +    exit({json_encode, {bad_term, Bad}});
>> +json_encode(Bad, State=#encoder{handler=Handler}) ->
>> +    json_encode(Handler(Bad), State).
>> +
>> +json_encode_array([], _State) ->
>> +    <<"[]">>;
>> +json_encode_array(L, State) ->
>> +    F = fun (O, Acc) ->
>> +                [$,, json_encode(O, State) | Acc]
>> +        end,
>> +    [$, | Acc1] = lists:foldl(F, "[", L),
>> +    lists:reverse([$\] | Acc1]).
>> +
>> +json_encode_proplist([], _State) ->
>> +    <<"{}">>;
>> +json_encode_proplist(Props, State) ->
>> +    F = fun ({K, V}, Acc) ->
>> +                KS = json_encode_string(K, State),
>> +                VS = json_encode(V, State),
>> +                [$,, VS, $:, KS | Acc]
>> +        end,
>> +    [$, | Acc1] = lists:foldl(F, "{", Props),
>> +    lists:reverse([$\} | Acc1]).
>> +
>> +json_encode_string(A, State) when is_atom(A) ->
>> +    L = atom_to_list(A),
>> +    case json_string_is_safe(L) of
>> +        true ->
>> +            [?Q, L, ?Q];
>> +        false ->
>> +            json_encode_string_unicode(xmerl_ucs:from_utf8(L), State, [?Q])
>> +    end;
>> +json_encode_string(B, State) when is_binary(B) ->
>> +    case json_bin_is_safe(B) of
>> +        true ->
>> +            [?Q, B, ?Q];
>> +        false ->
>> +            json_encode_string_unicode(xmerl_ucs:from_utf8(B), State, [?Q])
>> +    end;
>> +json_encode_string(I, _State) when is_integer(I) ->
>> +    [?Q, integer_to_list(I), ?Q];
>> +json_encode_string(L, State) when is_list(L) ->
>> +    case json_string_is_safe(L) of
>> +        true ->
>> +            [?Q, L, ?Q];
>> +        false ->
>> +            json_encode_string_unicode(L, State, [?Q])
>> +    end.
>> +
>> +json_string_is_safe([]) ->
>> +    true;
>> +json_string_is_safe([C | Rest]) ->
>> +    case C of
>> +        ?Q ->
>> +            false;
>> +        $\\ ->
>> +            false;
>> +        $\b ->
>> +            false;
>> +        $\f ->
>> +            false;
>> +        $\n ->
>> +            false;
>> +        $\r ->
>> +            false;
>> +        $\t ->
>> +            false;
>> +        C when C >= 0, C < $\s; C >= 16#7f, C =< 16#10FFFF ->
>> +            false;
>> +        C when C < 16#7f ->
>> +            json_string_is_safe(Rest);
>> +        _ ->
>> +            false
>> +    end.
>> +
>> +json_bin_is_safe(<<>>) ->
>> +    true;
>> +json_bin_is_safe(<<C, Rest/binary>>) ->
>> +    case C of
>> +        ?Q ->
>> +            false;
>> +        $\\ ->
>> +            false;
>> +        $\b ->
>> +            false;
>> +        $\f ->
>> +            false;
>> +        $\n ->
>> +            false;
>> +        $\r ->
>> +            false;
>> +        $\t ->
>> +            false;
>> +        C when C >= 0, C < $\s; C >= 16#7f ->
>> +            false;
>> +        C when C < 16#7f ->
>> +            json_bin_is_safe(Rest)
>> +    end.
>> +
>> +json_encode_string_unicode([], _State, Acc) ->
>> +    lists:reverse([$\" | Acc]);
>> +json_encode_string_unicode([C | Cs], State, Acc) ->
>> +    Acc1 = case C of
>> +               ?Q ->
>> +                   [?Q, $\\ | Acc];
>> +               %% Escaping solidus is only useful when trying to protect
>> +               %% against "</script>" injection attacks which are only
>> +               %% possible when JSON is inserted into a HTML document
>> +               %% in-line. mochijson2 does not protect you from this, so
>> +               %% if you do insert directly into HTML then you need to
>> +               %% uncomment the following case or escape the output of encode.
>> +               %%
>> +               %% $/ ->
>> +               %%    [$/, $\\ | Acc];
>> +               %%
>> +               $\\ ->
>> +                   [$\\, $\\ | Acc];
>> +               $\b ->
>> +                   [$b, $\\ | Acc];
>> +               $\f ->
>> +                   [$f, $\\ | Acc];
>> +               $\n ->
>> +                   [$n, $\\ | Acc];
>> +               $\r ->
>> +                   [$r, $\\ | Acc];
>> +               $\t ->
>> +                   [$t, $\\ | Acc];
>> +               C when C >= 0, C < $\s ->
>> +                   [unihex(C) | Acc];
>> +               C when C >= 16#7f, C =< 16#10FFFF, State#encoder.utf8 ->
>> +                   [xmerl_ucs:to_utf8(C) | Acc];
>> +               C when  C >= 16#7f, C =< 16#10FFFF, not State#encoder.utf8 ->
>> +                   [unihex(C) | Acc];
>> +               C when C < 16#7f ->
>> +                   [C | Acc];
>> +               _ ->
>> +                   exit({json_encode, {bad_char, C}})
>> +           end,
>> +    json_encode_string_unicode(Cs, State, Acc1).
>> +
>> +hexdigit(C) when C >= 0, C =< 9 ->
>> +    C + $0;
>> +hexdigit(C) when C =< 15 ->
>> +    C + $a - 10.
>> +
>> +unihex(C) when C < 16#10000 ->
>> +    <<D3:4, D2:4, D1:4, D0:4>> = <<C:16>>,
>> +    Digits = [hexdigit(D) || D <- [D3, D2, D1, D0]],
>> +    [$\\, $u | Digits];
>> +unihex(C) when C =< 16#10FFFF ->
>> +    N = C - 16#10000,
>> +    S1 = 16#d800 bor ((N bsr 10) band 16#3ff),
>> +    S2 = 16#dc00 bor (N band 16#3ff),
>> +    [unihex(S1), unihex(S2)].
>> +
>> +json_decode(L, S) when is_list(L) ->
>> +    json_decode(iolist_to_binary(L), S);
>> +json_decode(B, S) ->
>> +    {Res, S1} = decode1(B, S),
>> +    {eof, _} = tokenize(B, S1#decoder{state=trim}),
>> +    Res.
>> +
>> +decode1(B, S=#decoder{state=null}) ->
>> +    case tokenize(B, S#decoder{state=any}) of
>> +        {{const, C}, S1} ->
>> +            {C, S1};
>> +        {start_array, S1} ->
>> +            decode_array(B, S1);
>> +        {start_object, S1} ->
>> +            decode_object(B, S1)
>> +    end.
>> +
>> +make_object(V, #decoder{object_hook=null}) ->
>> +    V;
>> +make_object(V, #decoder{object_hook=Hook}) ->
>> +    Hook(V).
>> +
>> +decode_object(B, S) ->
>> +    decode_object(B, S#decoder{state=key}, []).
>> +
>> +decode_object(B, S=#decoder{state=key}, Acc) ->
>> +    case tokenize(B, S) of
>> +        {end_object, S1} ->
>> +            V = make_object({struct, lists:reverse(Acc)}, S1),
>> +            {V, S1#decoder{state=null}};
>> +        {{const, K}, S1} ->
>> +            {colon, S2} = tokenize(B, S1),
>> +            {V, S3} = decode1(B, S2#decoder{state=null}),
>> +            decode_object(B, S3#decoder{state=comma}, [{K, V} | Acc])
>> +    end;
>> +decode_object(B, S=#decoder{state=comma}, Acc) ->
>> +    case tokenize(B, S) of
>> +        {end_object, S1} ->
>> +            V = make_object({struct, lists:reverse(Acc)}, S1),
>> +            {V, S1#decoder{state=null}};
>> +        {comma, S1} ->
>> +            decode_object(B, S1#decoder{state=key}, Acc)
>> +    end.
>> +
>> +decode_array(B, S) ->
>> +    decode_array(B, S#decoder{state=any}, []).
>> +
>> +decode_array(B, S=#decoder{state=any}, Acc) ->
>> +    case tokenize(B, S) of
>> +        {end_array, S1} ->
>> +            {lists:reverse(Acc), S1#decoder{state=null}};
>> +        {start_array, S1} ->
>> +            {Array, S2} = decode_array(B, S1),
>> +            decode_array(B, S2#decoder{state=comma}, [Array | Acc]);
>> +        {start_object, S1} ->
>> +            {Array, S2} = decode_object(B, S1),
>> +            decode_array(B, S2#decoder{state=comma}, [Array | Acc]);
>> +        {{const, Const}, S1} ->
>> +            decode_array(B, S1#decoder{state=comma}, [Const | Acc])
>> +    end;
>> +decode_array(B, S=#decoder{state=comma}, Acc) ->
>> +    case tokenize(B, S) of
>> +        {end_array, S1} ->
>> +            {lists:reverse(Acc), S1#decoder{state=null}};
>> +        {comma, S1} ->
>> +            decode_array(B, S1#decoder{state=any}, Acc)
>> +    end.
>> +
>> +tokenize_string(B, S=#decoder{offset=O}) ->
>> +    case tokenize_string_fast(B, O) of
>> +        {escape, O1} ->
>> +            Length = O1 - O,
>> +            S1 = ?ADV_COL(S, Length),
>> +            <<_:O/binary, Head:Length/binary, _/binary>> = B,
>> +            tokenize_string(B, S1, lists:reverse(binary_to_list(Head)));
>> +        O1 ->
>> +            Length = O1 - O,
>> +            <<_:O/binary, String:Length/binary, ?Q, _/binary>> = B,
>> +            {{const, String}, ?ADV_COL(S, Length + 1)}
>> +    end.
>> +
>> +tokenize_string_fast(B, O) ->
>> +    case B of
>> +        <<_:O/binary, ?Q, _/binary>> ->
>> +            O;
>> +        <<_:O/binary, $\\, _/binary>> ->
>> +            {escape, O};
>> +        <<_:O/binary, C1, _/binary>> when C1 < 128 ->
>> +            tokenize_string_fast(B, 1 + O);
>> +        <<_:O/binary, C1, C2, _/binary>> when C1 >= 194, C1 =< 223,
>> +                C2 >= 128, C2 =< 191 ->
>> +            tokenize_string_fast(B, 2 + O);
>> +        <<_:O/binary, C1, C2, C3, _/binary>> when C1 >= 224, C1 =< 239,
>> +                C2 >= 128, C2 =< 191,
>> +                C3 >= 128, C3 =< 191 ->
>> +            tokenize_string_fast(B, 3 + O);
>> +        <<_:O/binary, C1, C2, C3, C4, _/binary>> when C1 >= 240, C1 =< 244,
>> +                C2 >= 128, C2 =< 191,
>> +                C3 >= 128, C3 =< 191,
>> +                C4 >= 128, C4 =< 191 ->
>> +            tokenize_string_fast(B, 4 + O);
>> +        _ ->
>> +            throw(invalid_utf8)
>> +    end.
>> +
>> +tokenize_string(B, S=#decoder{offset=O}, Acc) ->
>> +    case B of
>> +        <<_:O/binary, ?Q, _/binary>> ->
>> +            {{const, iolist_to_binary(lists:reverse(Acc))}, ?INC_COL(S)};
>> +        <<_:O/binary, "\\\"", _/binary>> ->
>> +            tokenize_string(B, ?ADV_COL(S, 2), [$\" | Acc]);
>> +        <<_:O/binary, "\\\\", _/binary>> ->
>> +            tokenize_string(B, ?ADV_COL(S, 2), [$\\ | Acc]);
>> +        <<_:O/binary, "\\/", _/binary>> ->
>> +            tokenize_string(B, ?ADV_COL(S, 2), [$/ | Acc]);
>> +        <<_:O/binary, "\\b", _/binary>> ->
>> +            tokenize_string(B, ?ADV_COL(S, 2), [$\b | Acc]);
>> +        <<_:O/binary, "\\f", _/binary>> ->
>> +            tokenize_string(B, ?ADV_COL(S, 2), [$\f | Acc]);
>> +        <<_:O/binary, "\\n", _/binary>> ->
>> +            tokenize_string(B, ?ADV_COL(S, 2), [$\n | Acc]);
>> +        <<_:O/binary, "\\r", _/binary>> ->
>> +            tokenize_string(B, ?ADV_COL(S, 2), [$\r | Acc]);
>> +        <<_:O/binary, "\\t", _/binary>> ->
>> +            tokenize_string(B, ?ADV_COL(S, 2), [$\t | Acc]);
>> +        <<_:O/binary, "\\u", C3, C2, C1, C0, Rest/binary>> ->
>> +            C = erlang:list_to_integer([C3, C2, C1, C0], 16),
>> +            if C > 16#D7FF, C < 16#DC00 ->
>> +                %% coalesce UTF-16 surrogate pair
>> +                <<"\\u", D3, D2, D1, D0, _/binary>> = Rest,
>> +                D = erlang:list_to_integer([D3,D2,D1,D0], 16),
>> +                [CodePoint] = xmerl_ucs:from_utf16be(<<C:16/big-unsigned-integer,
>> +                    D:16/big-unsigned-integer>>),
>> +                Acc1 = lists:reverse(xmerl_ucs:to_utf8(CodePoint), Acc),
>> +                tokenize_string(B, ?ADV_COL(S, 12), Acc1);
>> +            true ->
>> +                Acc1 = lists:reverse(xmerl_ucs:to_utf8(C), Acc),
>> +                tokenize_string(B, ?ADV_COL(S, 6), Acc1)
>> +            end;
>> +        <<_:O/binary, C1, _/binary>> when C1 < 128 ->
>> +            tokenize_string(B, ?INC_CHAR(S, C1), [C1 | Acc]);
>> +        <<_:O/binary, C1, C2, _/binary>> when C1 >= 194, C1 =< 223,
>> +                C2 >= 128, C2 =< 191 ->
>> +            tokenize_string(B, ?ADV_COL(S, 2), [C2, C1 | Acc]);
>> +        <<_:O/binary, C1, C2, C3, _/binary>> when C1 >= 224, C1 =< 239,
>> +                C2 >= 128, C2 =< 191,
>> +                C3 >= 128, C3 =< 191 ->
>> +            tokenize_string(B, ?ADV_COL(S, 3), [C3, C2, C1 | Acc]);
>> +        <<_:O/binary, C1, C2, C3, C4, _/binary>> when C1 >= 240, C1 =< 244,
>> +                C2 >= 128, C2 =< 191,
>> +                C3 >= 128, C3 =< 191,
>> +                C4 >= 128, C4 =< 191 ->
>> +            tokenize_string(B, ?ADV_COL(S, 4), [C4, C3, C2, C1 | Acc]);
>> +        _ ->
>> +            throw(invalid_utf8)
>> +    end.
>> +
>> +tokenize_number(B, S) ->
>> +    case tokenize_number(B, sign, S, []) of
>> +        {{int, Int}, S1} ->
>> +            {{const, list_to_integer(Int)}, S1};
>> +        {{float, Float}, S1} ->
>> +            {{const, list_to_float(Float)}, S1}
>> +    end.
>> +
>> +tokenize_number(B, sign, S=#decoder{offset=O}, []) ->
>> +    case B of
>> +        <<_:O/binary, $-, _/binary>> ->
>> +            tokenize_number(B, int, ?INC_COL(S), [$-]);
>> +        _ ->
>> +            tokenize_number(B, int, S, [])
>> +    end;
>> +tokenize_number(B, int, S=#decoder{offset=O}, Acc) ->
>> +    case B of
>> +        <<_:O/binary, $0, _/binary>> ->
>> +            tokenize_number(B, frac, ?INC_COL(S), [$0 | Acc]);
>> +        <<_:O/binary, C, _/binary>> when C >= $1 andalso C =< $9 ->
>> +            tokenize_number(B, int1, ?INC_COL(S), [C | Acc])
>> +    end;
>> +tokenize_number(B, int1, S=#decoder{offset=O}, Acc) ->
>> +    case B of
>> +        <<_:O/binary, C, _/binary>> when C >= $0 andalso C =< $9 ->
>> +            tokenize_number(B, int1, ?INC_COL(S), [C | Acc]);
>> +        _ ->
>> +            tokenize_number(B, frac, S, Acc)
>> +    end;
>> +tokenize_number(B, frac, S=#decoder{offset=O}, Acc) ->
>> +    case B of
>> +        <<_:O/binary, $., C, _/binary>> when C >= $0, C =< $9 ->
>> +            tokenize_number(B, frac1, ?ADV_COL(S, 2), [C, $. | Acc]);
>> +        <<_:O/binary, E, _/binary>> when E =:= $e orelse E =:= $E ->
>> +            tokenize_number(B, esign, ?INC_COL(S), [$e, $0, $. | Acc]);
>> +        _ ->
>> +            {{int, lists:reverse(Acc)}, S}
>> +    end;
>> +tokenize_number(B, frac1, S=#decoder{offset=O}, Acc) ->
>> +    case B of
>> +        <<_:O/binary, C, _/binary>> when C >= $0 andalso C =< $9 ->
>> +            tokenize_number(B, frac1, ?INC_COL(S), [C | Acc]);
>> +        <<_:O/binary, E, _/binary>> when E =:= $e orelse E =:= $E ->
>> +            tokenize_number(B, esign, ?INC_COL(S), [$e | Acc]);
>> +        _ ->
>> +            {{float, lists:reverse(Acc)}, S}
>> +    end;
>> +tokenize_number(B, esign, S=#decoder{offset=O}, Acc) ->
>> +    case B of
>> +        <<_:O/binary, C, _/binary>> when C =:= $- orelse C=:= $+ ->
>> +            tokenize_number(B, eint, ?INC_COL(S), [C | Acc]);
>> +        _ ->
>> +            tokenize_number(B, eint, S, Acc)
>> +    end;
>> +tokenize_number(B, eint, S=#decoder{offset=O}, Acc) ->
>> +    case B of
>> +        <<_:O/binary, C, _/binary>> when C >= $0 andalso C =< $9 ->
>> +            tokenize_number(B, eint1, ?INC_COL(S), [C | Acc])
>> +    end;
>> +tokenize_number(B, eint1, S=#decoder{offset=O}, Acc) ->
>> +    case B of
>> +        <<_:O/binary, C, _/binary>> when C >= $0 andalso C =< $9 ->
>> +            tokenize_number(B, eint1, ?INC_COL(S), [C | Acc]);
>> +        _ ->
>> +            {{float, lists:reverse(Acc)}, S}
>> +    end.
>> +
>> +tokenize(B, S=#decoder{offset=O}) ->
>> +    case B of
>> +        <<_:O/binary, C, _/binary>> when ?IS_WHITESPACE(C) ->
>> +            tokenize(B, ?INC_CHAR(S, C));
>> +        <<_:O/binary, "{", _/binary>> ->
>> +            {start_object, ?INC_COL(S)};
>> +        <<_:O/binary, "}", _/binary>> ->
>> +            {end_object, ?INC_COL(S)};
>> +        <<_:O/binary, "[", _/binary>> ->
>> +            {start_array, ?INC_COL(S)};
>> +        <<_:O/binary, "]", _/binary>> ->
>> +            {end_array, ?INC_COL(S)};
>> +        <<_:O/binary, ",", _/binary>> ->
>> +            {comma, ?INC_COL(S)};
>> +        <<_:O/binary, ":", _/binary>> ->
>> +            {colon, ?INC_COL(S)};
>> +        <<_:O/binary, "null", _/binary>> ->
>> +            {{const, null}, ?ADV_COL(S, 4)};
>> +        <<_:O/binary, "true", _/binary>> ->
>> +            {{const, true}, ?ADV_COL(S, 4)};
>> +        <<_:O/binary, "false", _/binary>> ->
>> +            {{const, false}, ?ADV_COL(S, 5)};
>> +        <<_:O/binary, "\"", _/binary>> ->
>> +            tokenize_string(B, ?INC_COL(S));
>> +        <<_:O/binary, C, _/binary>> when (C >= $0 andalso C =< $9)
>> +                                         orelse C =:= $- ->
>> +            tokenize_number(B, S);
>> +        <<_:O/binary>> ->
>> +            trim = S#decoder.state,
>> +            {eof, S}
>> +    end.
>> +%%
>> +%% Tests
>> +%%
>> +-ifdef(TEST).
>> +-include_lib("eunit/include/eunit.hrl").
>> +
>> +
>> +%% testing constructs borrowed from the Yaws JSON implementation.
>> +
>> +%% Create an object from a list of Key/Value pairs.
>> +
>> +obj_new() ->
>> +    {struct, []}.
>> +
>> +is_obj({struct, Props}) ->
>> +    F = fun ({K, _}) when is_binary(K) -> true end,
>> +    lists:all(F, Props).
>> +
>> +obj_from_list(Props) ->
>> +    Obj = {struct, Props},
>> +    ?assert(is_obj(Obj)),
>> +    Obj.
>> +
>> +%% Test for equivalence of Erlang terms.
>> +%% Due to arbitrary order of construction, equivalent objects might
>> +%% compare unequal as erlang terms, so we need to carefully recurse
>> +%% through aggregates (tuples and objects).
>> +
>> +equiv({struct, Props1}, {struct, Props2}) ->
>> +    equiv_object(Props1, Props2);
>> +equiv(L1, L2) when is_list(L1), is_list(L2) ->
>> +    equiv_list(L1, L2);
>> +equiv(N1, N2) when is_number(N1), is_number(N2) -> N1 == N2;
>> +equiv(B1, B2) when is_binary(B1), is_binary(B2) -> B1 == B2;
>> +equiv(A, A) when A =:= true orelse A =:= false orelse A =:= null -> true.
>> +
>> +%% Object representation and traversal order is unknown.
>> +%% Use the sledgehammer and sort property lists.
>> +
>> +equiv_object(Props1, Props2) ->
>> +    L1 = lists:keysort(1, Props1),
>> +    L2 = lists:keysort(1, Props2),
>> +    Pairs = lists:zip(L1, L2),
>> +    true = lists:all(fun({{K1, V1}, {K2, V2}}) ->
>> +                             equiv(K1, K2) and equiv(V1, V2)
>> +                     end, Pairs).
>> +
>> +%% Recursively compare tuple elements for equivalence.
>> +
>> +equiv_list([], []) ->
>> +    true;
>> +equiv_list([V1 | L1], [V2 | L2]) ->
>> +    equiv(V1, V2) andalso equiv_list(L1, L2).
>> +
>> +decode_test() ->
>> +    [1199344435545.0, 1] = decode(<<"[1199344435545.0,1]">>),
>> +    <<16#F0,16#9D,16#9C,16#95>> = decode([34,"\\ud835","\\udf15",34]).
>> +
>> +e2j_vec_test() ->
>> +    test_one(e2j_test_vec(utf8), 1).
>> +
>> +test_one([], _N) ->
>> +    %% io:format("~p tests passed~n", [N-1]),
>> +    ok;
>> +test_one([{E, J} | Rest], N) ->
>> +    %% io:format("[~p] ~p ~p~n", [N, E, J]),
>> +    true = equiv(E, decode(J)),
>> +    true = equiv(E, decode(encode(E))),
>> +    test_one(Rest, 1+N).
>> +
>> +e2j_test_vec(utf8) ->
>> +    [
>> +     {1, "1"},
>> +     {3.1416, "3.14160"}, %% text representation may truncate, trail zeroes
>> +     {-1, "-1"},
>> +     {-3.1416, "-3.14160"},
>> +     {12.0e10, "1.20000e+11"},
>> +     {1.234E+10, "1.23400e+10"},
>> +     {-1.234E-10, "-1.23400e-10"},
>> +     {10.0, "1.0e+01"},
>> +     {123.456, "1.23456E+2"},
>> +     {10.0, "1e1"},
>> +     {<<"foo">>, "\"foo\""},
>> +     {<<"foo", 5, "bar">>, "\"foo\\u0005bar\""},
>> +     {<<"">>, "\"\""},
>> +     {<<"\n\n\n">>, "\"\\n\\n\\n\""},
>> +     {<<"\" \b\f\r\n\t\"">>, "\"\\\" \\b\\f\\r\\n\\t\\\"\""},
>> +     {obj_new(), "{}"},
>> +     {obj_from_list([{<<"foo">>, <<"bar">>}]), "{\"foo\":\"bar\"}"},
>> +     {obj_from_list([{<<"foo">>, <<"bar">>}, {<<"baz">>, 123}]),
>> +      "{\"foo\":\"bar\",\"baz\":123}"},
>> +     {[], "[]"},
>> +     {[[]], "[[]]"},
>> +     {[1, <<"foo">>], "[1,\"foo\"]"},
>> +
>> +     %% json array in a json object
>> +     {obj_from_list([{<<"foo">>, [123]}]),
>> +      "{\"foo\":[123]}"},
>> +
>> +     %% json object in a json object
>> +     {obj_from_list([{<<"foo">>, obj_from_list([{<<"bar">>, true}])}]),
>> +      "{\"foo\":{\"bar\":true}}"},
>> +
>> +     %% fold evaluation order
>> +     {obj_from_list([{<<"foo">>, []},
>> +                     {<<"bar">>, obj_from_list([{<<"baz">>, true}])},
>> +                     {<<"alice">>, <<"bob">>}]),
>> +      "{\"foo\":[],\"bar\":{\"baz\":true},\"alice\":\"bob\"}"},
>> +
>> +     %% json object in a json array
>> +     {[-123, <<"foo">>, obj_from_list([{<<"bar">>, []}]), null],
>> +      "[-123,\"foo\",{\"bar\":[]},null]"}
>> +    ].
>> +
>> +%% test utf8 encoding
>> +encoder_utf8_test() ->
>> +    %% safe conversion case (default)
>> +    [34,"\\u0001","\\u0442","\\u0435","\\u0441","\\u0442",34] =
>> +        encode(<<1,"\321\202\320\265\321\201\321\202">>),
>> +
>> +    %% raw utf8 output (optional)
>> +    Enc = mochijson2:encoder([{utf8, true}]),
>> +    [34,"\\u0001",[209,130],[208,181],[209,129],[209,130],34] =
>> +        Enc(<<1,"\321\202\320\265\321\201\321\202">>).
>> +
>> +input_validation_test() ->
>> +    Good = [
>> +        {16#00A3, <<?Q, 16#C2, 16#A3, ?Q>>}, %% pound
>> +        {16#20AC, <<?Q, 16#E2, 16#82, 16#AC, ?Q>>}, %% euro
>> +        {16#10196, <<?Q, 16#F0, 16#90, 16#86, 16#96, ?Q>>} %% denarius
>> +    ],
>> +    lists:foreach(fun({CodePoint, UTF8}) ->
>> +        Expect = list_to_binary(xmerl_ucs:to_utf8(CodePoint)),
>> +        Expect = decode(UTF8)
>> +    end, Good),
>> +
>> +    Bad = [
>> +        %% 2nd, 3rd, or 4th byte of a multi-byte sequence w/o leading byte
>> +        <<?Q, 16#80, ?Q>>,
>> +        %% missing continuations, last byte in each should be 80-BF
>> +        <<?Q, 16#C2, 16#7F, ?Q>>,
>> +        <<?Q, 16#E0, 16#80,16#7F, ?Q>>,
>> +        <<?Q, 16#F0, 16#80, 16#80, 16#7F, ?Q>>,
>> +        %% we don't support code points > 10FFFF per RFC 3629
>> +        <<?Q, 16#F5, 16#80, 16#80, 16#80, ?Q>>,
>> +        %% escape characters trigger a different code path
>> +        <<?Q, $\\, $\n, 16#80, ?Q>>
>> +    ],
>> +    lists:foreach(
>> +      fun(X) ->
>> +              ok = try decode(X) catch invalid_utf8 -> ok end,
>> +              %% could be {ucs,{bad_utf8_character_code}} or
>> +              %%          {json_encode,{bad_char,_}}
>> +              {'EXIT', _} = (catch encode(X))
>> +      end, Bad).
>> +
>> +inline_json_test() ->
>> +    ?assertEqual(<<"\"iodata iodata\"">>,
>> +                 iolist_to_binary(
>> +                   encode({json, [<<"\"iodata">>, " iodata\""]}))),
>> +    ?assertEqual({struct, [{<<"key">>, <<"iodata iodata">>}]},
>> +                 decode(
>> +                   encode({struct,
>> +                           [{key, {json, [<<"\"iodata">>, " iodata\""]}}]}))),
>> +    ok.
>> +
>> +big_unicode_test() ->
>> +    UTF8Seq = list_to_binary(xmerl_ucs:to_utf8(16#0001d120)),
>> +    ?assertEqual(
>> +       <<"\"\\ud834\\udd20\"">>,
>> +       iolist_to_binary(encode(UTF8Seq))),
>> +    ?assertEqual(
>> +       UTF8Seq,
>> +       decode(iolist_to_binary(encode(UTF8Seq)))),
>> +    ok.
>> +
>> +custom_decoder_test() ->
>> +    ?assertEqual(
>> +       {struct, [{<<"key">>, <<"value">>}]},
>> +       (decoder([]))("{\"key\": \"value\"}")),
>> +    F = fun ({struct, [{<<"key">>, <<"value">>}]}) -> win end,
>> +    ?assertEqual(
>> +       win,
>> +       (decoder([{object_hook, F}]))("{\"key\": \"value\"}")),
>> +    ok.
>> +
>> +atom_test() ->
>> +    %% JSON native atoms
>> +    [begin
>> +         ?assertEqual(A, decode(atom_to_list(A))),
>> +         ?assertEqual(iolist_to_binary(atom_to_list(A)),
>> +                      iolist_to_binary(encode(A)))
>> +     end || A <- [true, false, null]],
>> +    %% Atom to string
>> +    ?assertEqual(
>> +       <<"\"foo\"">>,
>> +       iolist_to_binary(encode(foo))),
>> +    ?assertEqual(
>> +       <<"\"\\ud834\\udd20\"">>,
>> +       iolist_to_binary(encode(list_to_atom(xmerl_ucs:to_utf8(16#0001d120))))),
>> +    ok.
>> +
>> +key_encode_test() ->
>> +    %% Some forms are accepted as keys that would not be strings in other
>> +    %% cases
>> +    ?assertEqual(
>> +       <<"{\"foo\":1}">>,
>> +       iolist_to_binary(encode({struct, [{foo, 1}]}))),
>> +    ?assertEqual(
>> +       <<"{\"foo\":1}">>,
>> +       iolist_to_binary(encode({struct, [{<<"foo">>, 1}]}))),
>> +    ?assertEqual(
>> +       <<"{\"foo\":1}">>,
>> +       iolist_to_binary(encode({struct, [{"foo", 1}]}))),
>> +       ?assertEqual(
>> +       <<"{\"foo\":1}">>,
>> +       iolist_to_binary(encode([{foo, 1}]))),
>> +    ?assertEqual(
>> +       <<"{\"foo\":1}">>,
>> +       iolist_to_binary(encode([{<<"foo">>, 1}]))),
>> +    ?assertEqual(
>> +       <<"{\"foo\":1}">>,
>> +       iolist_to_binary(encode([{"foo", 1}]))),
>> +    ?assertEqual(
>> +       <<"{\"\\ud834\\udd20\":1}">>,
>> +       iolist_to_binary(
>> +         encode({struct, [{[16#0001d120], 1}]}))),
>> +    ?assertEqual(
>> +       <<"{\"1\":1}">>,
>> +       iolist_to_binary(encode({struct, [{1, 1}]}))),
>> +    ok.
>> +
>> +unsafe_chars_test() ->
>> +    Chars = "\"\\\b\f\n\r\t",
>> +    [begin
>> +         ?assertEqual(false, json_string_is_safe([C])),
>> +         ?assertEqual(false, json_bin_is_safe(<<C>>)),
>> +         ?assertEqual(<<C>>, decode(encode(<<C>>)))
>> +     end || C <- Chars],
>> +    ?assertEqual(
>> +       false,
>> +       json_string_is_safe([16#0001d120])),
>> +    ?assertEqual(
>> +       false,
>> +       json_bin_is_safe(list_to_binary(xmerl_ucs:to_utf8(16#0001d120)))),
>> +    ?assertEqual(
>> +       [16#0001d120],
>> +       xmerl_ucs:from_utf8(
>> +         binary_to_list(
>> +           decode(encode(list_to_atom(xmerl_ucs:to_utf8(16#0001d120))))))),
>> +    ?assertEqual(
>> +       false,
>> +       json_string_is_safe([16#110000])),
>> +    ?assertEqual(
>> +       false,
>> +       json_bin_is_safe(list_to_binary(xmerl_ucs:to_utf8([16#110000])))),
>> +    %% solidus can be escaped but isn't unsafe by default
>> +    ?assertEqual(
>> +       <<"/">>,
>> +       decode(<<"\"\\/\"">>)),
>> +    ok.
>> +
>> +int_test() ->
>> +    ?assertEqual(0, decode("0")),
>> +    ?assertEqual(1, decode("1")),
>> +    ?assertEqual(11, decode("11")),
>> +    ok.
>> +
>> +large_int_test() ->
>> +    ?assertEqual(<<"-2147483649214748364921474836492147483649">>,
>> +        iolist_to_binary(encode(-2147483649214748364921474836492147483649))),
>> +    ?assertEqual(<<"2147483649214748364921474836492147483649">>,
>> +        iolist_to_binary(encode(2147483649214748364921474836492147483649))),
>> +    ok.
>> +
>> +float_test() ->
>> +    ?assertEqual(<<"-2147483649.0">>, iolist_to_binary(encode(-2147483649.0))),
>> +    ?assertEqual(<<"2147483648.0">>, iolist_to_binary(encode(2147483648.0))),
>> +    ok.
>> +
>> +handler_test() ->
>> +    ?assertEqual(
>> +       {'EXIT',{json_encode,{bad_term,{}}}},
>> +       catch encode({})),
>> +    F = fun ({}) -> [] end,
>> +    ?assertEqual(
>> +       <<"[]">>,
>> +       iolist_to_binary((encoder([{handler, F}]))({}))),
>> +    ok.
>> +
>> +-endif.
>>
>> Added: couchdb/trunk/src/ejson/mochinum.erl
>> URL: http://svn.apache.org/viewvc/couchdb/trunk/src/ejson/mochinum.erl?rev=1088941&view=auto
>> ==============================================================================
>> --- couchdb/trunk/src/ejson/mochinum.erl (added)
>> +++ couchdb/trunk/src/ejson/mochinum.erl Tue Apr  5 09:42:41 2011
>> @@ -0,0 +1,354 @@
>> +%% @copyright 2007 Mochi Media, Inc.
>> +%% @author Bob Ippolito <bob@mochimedia.com>
>> +
>> +%% @doc Useful numeric algorithms for floats that cover some deficiencies
>> +%% in the math module. More interesting is digits/1, which implements
>> +%% the algorithm from:
>> +%% http://www.cs.indiana.edu/~burger/fp/index.html
>> +%% See also "Printing Floating-Point Numbers Quickly and Accurately"
>> +%% in Proceedings of the SIGPLAN '96 Conference on Programming Language
>> +%% Design and Implementation.
>> +
>> +-module(mochinum).
>> +-author("Bob Ippolito <bob@mochimedia.com>").
>> +-export([digits/1, frexp/1, int_pow/2, int_ceil/1]).
>> +
>> +%% IEEE 754 Float exponent bias
>> +-define(FLOAT_BIAS, 1022).
>> +-define(MIN_EXP, -1074).
>> +-define(BIG_POW, 4503599627370496).
>> +
>> +%% External API
>> +
>> +%% @spec digits(number()) -> string()
>> +%% @doc  Returns a string that accurately represents the given integer or float
>> +%%       using a conservative amount of digits. Great for generating
>> +%%       human-readable output, or compact ASCII serializations for floats.
>> +digits(N) when is_integer(N) ->
>> +    integer_to_list(N);
>> +digits(0.0) ->
>> +    "0.0";
>> +digits(Float) ->
>> +    {Frac1, Exp1} = frexp_int(Float),
>> +    [Place0 | Digits0] = digits1(Float, Exp1, Frac1),
>> +    {Place, Digits} = transform_digits(Place0, Digits0),
>> +    R = insert_decimal(Place, Digits),
>> +    case Float < 0 of
>> +        true ->
>> +            [$- | R];
>> +        _ ->
>> +            R
>> +    end.
>> +
>> +%% @spec frexp(F::float()) -> {Frac::float(), Exp::float()}
>> +%% @doc  Return the fractional and exponent part of an IEEE 754 double,
>> +%%       equivalent to the libc function of the same name.
>> +%%       F = Frac * pow(2, Exp).
>> +frexp(F) ->
>> +    frexp1(unpack(F)).
>> +
>> +%% @spec int_pow(X::integer(), N::integer()) -> Y::integer()
>> +%% @doc  Moderately efficient way to exponentiate integers.
>> +%%       int_pow(10, 2) = 100.
>> +int_pow(_X, 0) ->
>> +    1;
>> +int_pow(X, N) when N > 0 ->
>> +    int_pow(X, N, 1).
>> +
>> +%% @spec int_ceil(F::float()) -> integer()
>> +%% @doc  Return the ceiling of F as an integer. The ceiling is defined as
>> +%%       F when F == trunc(F);
>> +%%       trunc(F) when F &lt; 0;
>> +%%       trunc(F) + 1 when F &gt; 0.
>> +int_ceil(X) ->
>> +    T = trunc(X),
>> +    case (X - T) of
>> +        Pos when Pos > 0 -> T + 1;
>> +        _ -> T
>> +    end.
>> +
>> +
>> +%% Internal API
>> +
>> +int_pow(X, N, R) when N < 2 ->
>> +    R * X;
>> +int_pow(X, N, R) ->
>> +    int_pow(X * X, N bsr 1, case N band 1 of 1 -> R * X; 0 -> R end).
>> +
>> +insert_decimal(0, S) ->
>> +    "0." ++ S;
>> +insert_decimal(Place, S) when Place > 0 ->
>> +    L = length(S),
>> +    case Place - L of
>> +         0 ->
>> +            S ++ ".0";
>> +        N when N < 0 ->
>> +            {S0, S1} = lists:split(L + N, S),
>> +            S0 ++ "." ++ S1;
>> +        N when N < 6 ->
>> +            %% More places than digits
>> +            S ++ lists:duplicate(N, $0) ++ ".0";
>> +        _ ->
>> +            insert_decimal_exp(Place, S)
>> +    end;
>> +insert_decimal(Place, S) when Place > -6 ->
>> +    "0." ++ lists:duplicate(abs(Place), $0) ++ S;
>> +insert_decimal(Place, S) ->
>> +    insert_decimal_exp(Place, S).
>> +
>> +insert_decimal_exp(Place, S) ->
>> +    [C | S0] = S,
>> +    S1 = case S0 of
>> +             [] ->
>> +                 "0";
>> +             _ ->
>> +                 S0
>> +         end,
>> +    Exp = case Place < 0 of
>> +              true ->
>> +                  "e-";
>> +              false ->
>> +                  "e+"
>> +          end,
>> +    [C] ++ "." ++ S1 ++ Exp ++ integer_to_list(abs(Place - 1)).
>> +
>> +
>> +digits1(Float, Exp, Frac) ->
>> +    Round = ((Frac band 1) =:= 0),
>> +    case Exp >= 0 of
>> +        true ->
>> +            BExp = 1 bsl Exp,
>> +            case (Frac =/= ?BIG_POW) of
>> +                true ->
>> +                    scale((Frac * BExp * 2), 2, BExp, BExp,
>> +                          Round, Round, Float);
>> +                false ->
>> +                    scale((Frac * BExp * 4), 4, (BExp * 2), BExp,
>> +                          Round, Round, Float)
>> +            end;
>> +        false ->
>> +            case (Exp =:= ?MIN_EXP) orelse (Frac =/= ?BIG_POW) of
>> +                true ->
>> +                    scale((Frac * 2), 1 bsl (1 - Exp), 1, 1,
>> +                          Round, Round, Float);
>> +                false ->
>> +                    scale((Frac * 4), 1 bsl (2 - Exp), 2, 1,
>> +                          Round, Round, Float)
>> +            end
>> +    end.
>> +
>> +scale(R, S, MPlus, MMinus, LowOk, HighOk, Float) ->
>> +    Est = int_ceil(math:log10(abs(Float)) - 1.0e-10),
>> +    %% Note that the scheme implementation uses a 326 element look-up table
>> +    %% for int_pow(10, N) where we do not.
>> +    case Est >= 0 of
>> +        true ->
>> +            fixup(R, S * int_pow(10, Est), MPlus, MMinus, Est,
>> +                  LowOk, HighOk);
>> +        false ->
>> +            Scale = int_pow(10, -Est),
>> +            fixup(R * Scale, S, MPlus * Scale, MMinus * Scale, Est,
>> +                  LowOk, HighOk)
>> +    end.
>> +
>> +fixup(R, S, MPlus, MMinus, K, LowOk, HighOk) ->
>> +    TooLow = case HighOk of
>> +                 true ->
>> +                     (R + MPlus) >= S;
>> +                 false ->
>> +                     (R + MPlus) > S
>> +             end,
>> +    case TooLow of
>> +        true ->
>> +            [(K + 1) | generate(R, S, MPlus, MMinus, LowOk, HighOk)];
>> +        false ->
>> +            [K | generate(R * 10, S, MPlus * 10, MMinus * 10, LowOk, HighOk)]
>> +    end.
>> +
>> +generate(R0, S, MPlus, MMinus, LowOk, HighOk) ->
>> +    D = R0 div S,
>> +    R = R0 rem S,
>> +    TC1 = case LowOk of
>> +              true ->
>> +                  R =< MMinus;
>> +              false ->
>> +                  R < MMinus
>> +          end,
>> +    TC2 = case HighOk of
>> +              true ->
>> +                  (R + MPlus) >= S;
>> +              false ->
>> +                  (R + MPlus) > S
>> +          end,
>> +    case TC1 of
>> +        false ->
>> +            case TC2 of
>> +                false ->
>> +                    [D | generate(R * 10, S, MPlus * 10, MMinus * 10,
>> +                                  LowOk, HighOk)];
>> +                true ->
>> +                    [D + 1]
>> +            end;
>> +        true ->
>> +            case TC2 of
>> +                false ->
>> +                    [D];
>> +                true ->
>> +                    case R * 2 < S of
>> +                        true ->
>> +                            [D];
>> +                        false ->
>> +                            [D + 1]
>> +                    end
>> +            end
>> +    end.
>> +
>> +unpack(Float) ->
>> +    <<Sign:1, Exp:11, Frac:52>> = <<Float:64/float>>,
>> +    {Sign, Exp, Frac}.
>> +
>> +frexp1({_Sign, 0, 0}) ->
>> +    {0.0, 0};
>> +frexp1({Sign, 0, Frac}) ->
>> +    Exp = log2floor(Frac),
>> +    <<Frac1:64/float>> = <<Sign:1, ?FLOAT_BIAS:11, (Frac-1):52>>,
>> +    {Frac1, -(?FLOAT_BIAS) - 52 + Exp};
>> +frexp1({Sign, Exp, Frac}) ->
>> +    <<Frac1:64/float>> = <<Sign:1, ?FLOAT_BIAS:11, Frac:52>>,
>> +    {Frac1, Exp - ?FLOAT_BIAS}.
>> +
>> +log2floor(Int) ->
>> +    log2floor(Int, 0).
>> +
>> +log2floor(0, N) ->
>> +    N;
>> +log2floor(Int, N) ->
>> +    log2floor(Int bsr 1, 1 + N).
>> +
>> +
>> +transform_digits(Place, [0 | Rest]) ->
>> +    transform_digits(Place, Rest);
>> +transform_digits(Place, Digits) ->
>> +    {Place, [$0 + D || D <- Digits]}.
>> +
>> +
>> +frexp_int(F) ->
>> +    case unpack(F) of
>> +        {_Sign, 0, Frac} ->
>> +            {Frac, ?MIN_EXP};
>> +        {_Sign, Exp, Frac} ->
>> +            {Frac + (1 bsl 52), Exp - 53 - ?FLOAT_BIAS}
>> +    end.
>> +
>> +%%
>> +%% Tests
>> +%%
>> +-ifdef(TEST).
>> +-include_lib("eunit/include/eunit.hrl").
>> +
>> +int_ceil_test() ->
>> +    ?assertEqual(1, int_ceil(0.0001)),
>> +    ?assertEqual(0, int_ceil(0.0)),
>> +    ?assertEqual(1, int_ceil(0.99)),
>> +    ?assertEqual(1, int_ceil(1.0)),
>> +    ?assertEqual(-1, int_ceil(-1.5)),
>> +    ?assertEqual(-2, int_ceil(-2.0)),
>> +    ok.
>> +
>> +int_pow_test() ->
>> +    ?assertEqual(1, int_pow(1, 1)),
>> +    ?assertEqual(1, int_pow(1, 0)),
>> +    ?assertEqual(1, int_pow(10, 0)),
>> +    ?assertEqual(10, int_pow(10, 1)),
>> +    ?assertEqual(100, int_pow(10, 2)),
>> +    ?assertEqual(1000, int_pow(10, 3)),
>> +    ok.
>> +
>> +digits_test() ->
>> +    ?assertEqual("0",
>> +                 digits(0)),
>> +    ?assertEqual("0.0",
>> +                 digits(0.0)),
>> +    ?assertEqual("1.0",
>> +                 digits(1.0)),
>> +    ?assertEqual("-1.0",
>> +                 digits(-1.0)),
>> +    ?assertEqual("0.1",
>> +                 digits(0.1)),
>> +    ?assertEqual("0.01",
>> +                 digits(0.01)),
>> +    ?assertEqual("0.001",
>> +                 digits(0.001)),
>> +    ?assertEqual("1.0e+6",
>> +                 digits(1000000.0)),
>> +    ?assertEqual("0.5",
>> +                 digits(0.5)),
>> +    ?assertEqual("4503599627370496.0",
>> +                 digits(4503599627370496.0)),
>> +    %% small denormalized number
>> +    %% 4.94065645841246544177e-324 =:= 5.0e-324
>> +    <<SmallDenorm/float>> = <<0,0,0,0,0,0,0,1>>,
>> +    ?assertEqual("5.0e-324",
>> +                 digits(SmallDenorm)),
>> +    ?assertEqual(SmallDenorm,
>> +                 list_to_float(digits(SmallDenorm))),
>> +    %% large denormalized number
>> +    %% 2.22507385850720088902e-308
>> +    <<BigDenorm/float>> = <<0,15,255,255,255,255,255,255>>,
>> +    ?assertEqual("2.225073858507201e-308",
>> +                 digits(BigDenorm)),
>> +    ?assertEqual(BigDenorm,
>> +                 list_to_float(digits(BigDenorm))),
>> +    %% small normalized number
>> +    %% 2.22507385850720138309e-308
>> +    <<SmallNorm/float>> = <<0,16,0,0,0,0,0,0>>,
>> +    ?assertEqual("2.2250738585072014e-308",
>> +                 digits(SmallNorm)),
>> +    ?assertEqual(SmallNorm,
>> +                 list_to_float(digits(SmallNorm))),
>> +    %% large normalized number
>> +    %% 1.79769313486231570815e+308
>> +    <<LargeNorm/float>> = <<127,239,255,255,255,255,255,255>>,
>> +    ?assertEqual("1.7976931348623157e+308",
>> +                 digits(LargeNorm)),
>> +    ?assertEqual(LargeNorm,
>> +                 list_to_float(digits(LargeNorm))),
>> +    %% issue #10 - mochinum:frexp(math:pow(2, -1074)).
>> +    ?assertEqual("5.0e-324",
>> +                 digits(math:pow(2, -1074))),
>> +    ok.
>> +
>> +frexp_test() ->
>> +    %% zero
>> +    ?assertEqual({0.0, 0}, frexp(0.0)),
>> +    %% one
>> +    ?assertEqual({0.5, 1}, frexp(1.0)),
>> +    %% negative one
>> +    ?assertEqual({-0.5, 1}, frexp(-1.0)),
>> +    %% small denormalized number
>> +    %% 4.94065645841246544177e-324
>> +    <<SmallDenorm/float>> = <<0,0,0,0,0,0,0,1>>,
>> +    ?assertEqual({0.5, -1073}, frexp(SmallDenorm)),
>> +    %% large denormalized number
>> +    %% 2.22507385850720088902e-308
>> +    <<BigDenorm/float>> = <<0,15,255,255,255,255,255,255>>,
>> +    ?assertEqual(
>> +       {0.99999999999999978, -1022},
>> +       frexp(BigDenorm)),
>> +    %% small normalized number
>> +    %% 2.22507385850720138309e-308
>> +    <<SmallNorm/float>> = <<0,16,0,0,0,0,0,0>>,
>> +    ?assertEqual({0.5, -1021}, frexp(SmallNorm)),
>> +    %% large normalized number
>> +    %% 1.79769313486231570815e+308
>> +    <<LargeNorm/float>> = <<127,239,255,255,255,255,255,255>>,
>> +    ?assertEqual(
>> +        {0.99999999999999989, 1024},
>> +        frexp(LargeNorm)),
>> +    %% issue #10 - mochinum:frexp(math:pow(2, -1074)).
>> +    ?assertEqual(
>> +       {0.5, -1073},
>> +       frexp(math:pow(2, -1074))),
>> +    ok.
>> +
>> +-endif.
>>
>> Added: couchdb/trunk/src/ejson/yajl/yajl.c
>> URL: http://svn.apache.org/viewvc/couchdb/trunk/src/ejson/yajl/yajl.c?rev=1088941&view=auto
>> ==============================================================================
>> --- couchdb/trunk/src/ejson/yajl/yajl.c (added)
>> +++ couchdb/trunk/src/ejson/yajl/yajl.c Tue Apr  5 09:42:41 2011
>> @@ -0,0 +1,159 @@
>> +/*
>> + * Copyright 2010, Lloyd Hilaiel.
>> + *
>> + * Redistribution and use in source and binary forms, with or without
>> + * modification, are permitted provided that the following conditions are
>> + * met:
>> + *
>> + *  1. Redistributions of source code must retain the above copyright
>> + *     notice, this list of conditions and the following disclaimer.
>> + *
>> + *  2. Redistributions in binary form must reproduce the above copyright
>> + *     notice, this list of conditions and the following disclaimer in
>> + *     the documentation and/or other materials provided with the
>> + *     distribution.
>> + *
>> + *  3. Neither the name of Lloyd Hilaiel nor the names of its
>> + *     contributors may be used to endorse or promote products derived
>> + *     from this software without specific prior written permission.
>> + *
>> + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
>> + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
>> + * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
>> + * DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT,
>> + * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
>> + * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
>> + * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
>> + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
>> + * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
>> + * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
>> + * POSSIBILITY OF SUCH DAMAGE.
>> + */
>> +
>> +#include "yajl_parse.h"
>> +#include "yajl_lex.h"
>> +#include "yajl_parser.h"
>> +#include "yajl_alloc.h"
>> +
>> +#include <stdlib.h>
>> +#include <string.h>
>> +#include <assert.h>
>> +
>> +const char *
>> +yajl_status_to_string(yajl_status stat)
>> +{
>> +    const char * statStr = "unknown";
>> +    switch (stat) {
>> +        case yajl_status_ok:
>> +            statStr = "ok, no error";
>> +            break;
>> +        case yajl_status_client_canceled:
>> +            statStr = "client canceled parse";
>> +            break;
>> +        case yajl_status_insufficient_data:
>> +            statStr = "eof was met before the parse could complete";
>> +            break;
>> +        case yajl_status_error:
>> +            statStr = "parse error";
>> +            break;
>> +    }
>> +    return statStr;
>> +}
>> +
>> +yajl_handle
>> +yajl_alloc(const yajl_callbacks * callbacks,
>> +           const yajl_parser_config * config,
>> +           const yajl_alloc_funcs * afs,
>> +           void * ctx)
>> +{
>> +    unsigned int allowComments = 0;
>> +    unsigned int validateUTF8 = 0;
>> +    yajl_handle hand = NULL;
>> +    yajl_alloc_funcs afsBuffer;
>> +
>> +    /* first order of business is to set up memory allocation routines */
>> +    if (afs != NULL) {
>> +        if (afs->malloc == NULL || afs->realloc == NULL || afs->free == NULL)
>> +        {
>> +            return NULL;
>> +        }
>> +    } else {
>> +        yajl_set_default_alloc_funcs(&afsBuffer);
>> +        afs = &afsBuffer;
>> +    }
>> +
>> +    hand = (yajl_handle) YA_MALLOC(afs, sizeof(struct yajl_handle_t));
>> +
>> +    /* copy in pointers to allocation routines */
>> +    memcpy((void *) &(hand->alloc), (void *) afs, sizeof(yajl_alloc_funcs));
>> +
>> +    if (config != NULL) {
>> +        allowComments = config->allowComments;
>> +        validateUTF8 = config->checkUTF8;
>> +    }
>> +
>> +    hand->callbacks = callbacks;
>> +    hand->ctx = ctx;
>> +    hand->lexer = yajl_lex_alloc(&(hand->alloc), allowComments, validateUTF8);
>> +    hand->bytesConsumed = 0;
>> +    hand->decodeBuf = yajl_buf_alloc(&(hand->alloc));
>> +    yajl_bs_init(hand->stateStack, &(hand->alloc));
>> +
>> +    yajl_bs_push(hand->stateStack, yajl_state_start);
>> +
>> +    return hand;
>> +}
>> +
>> +void
>> +yajl_free(yajl_handle handle)
>> +{
>> +    yajl_bs_free(handle->stateStack);
>> +    yajl_buf_free(handle->decodeBuf);
>> +    yajl_lex_free(handle->lexer);
>> +    YA_FREE(&(handle->alloc), handle);
>> +}
>> +
>> +yajl_status
>> +yajl_parse(yajl_handle hand, const unsigned char * jsonText,
>> +           unsigned int jsonTextLen)
>> +{
>> +    yajl_status status;
>> +    status = yajl_do_parse(hand, jsonText, jsonTextLen);
>> +    return status;
>> +}
>> +
>> +yajl_status
>> +yajl_parse_complete(yajl_handle hand)
>> +{
>> +    /* The particular case we want to handle is a trailing number.
>> +     * Further input consisting of digits could cause our interpretation
>> +     * of the number to change (buffered "1" but "2" comes in).
>> +     * A very simple approach to this is to inject whitespace to terminate
>> +     * any number in the lex buffer.
>> +     */
>> +    return yajl_parse(hand, (const unsigned char *)" ", 1);
>> +}
>> +
>> +unsigned char *
>> +yajl_get_error(yajl_handle hand, int verbose,
>> +               const unsigned char * jsonText, unsigned int jsonTextLen)
>> +{
>> +    return yajl_render_error_string(hand, jsonText, jsonTextLen, verbose);
>> +}
>> +
>> +unsigned int
>> +yajl_get_bytes_consumed(yajl_handle hand)
>> +{
>> +    if (!hand) return 0;
>> +    else return hand->bytesConsumed;
>> +}
>> +
>> +
>> +void
>> +yajl_free_error(yajl_handle hand, unsigned char * str)
>> +{
>> +    /* use memory allocation functions if set */
>> +    YA_FREE(&(hand->alloc), str);
>> +}
>> +
>> +/* XXX: add utility routines to parse from file */
>>
>> Added: couchdb/trunk/src/ejson/yajl/yajl_alloc.c
>> URL: http://svn.apache.org/viewvc/couchdb/trunk/src/ejson/yajl/yajl_alloc.c?rev=1088941&view=auto
>> ==============================================================================
>> --- couchdb/trunk/src/ejson/yajl/yajl_alloc.c (added)
>> +++ couchdb/trunk/src/ejson/yajl/yajl_alloc.c Tue Apr  5 09:42:41 2011
>> @@ -0,0 +1,65 @@
>> +/*
>> + * Copyright 2010, Lloyd Hilaiel.
>> + *
>> + * Redistribution and use in source and binary forms, with or without
>> + * modification, are permitted provided that the following conditions are
>> + * met:
>> + *
>> + *  1. Redistributions of source code must retain the above copyright
>> + *     notice, this list of conditions and the following disclaimer.
>> + *
>> + *  2. Redistributions in binary form must reproduce the above copyright
>> + *     notice, this list of conditions and the following disclaimer in
>> + *     the documentation and/or other materials provided with the
>> + *     distribution.
>> + *
>> + *  3. Neither the name of Lloyd Hilaiel nor the names of its
>> + *     contributors may be used to endorse or promote products derived
>> + *     from this software without specific prior written permission.
>> + *
>> + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
>> + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
>> + * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
>> + * DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT,
>> + * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
>> + * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
>> + * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
>> + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
>> + * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
>> + * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
>> + * POSSIBILITY OF SUCH DAMAGE.
>> + */
>> +
>> +/**
>> + * \file yajl_alloc.h
>> + * default memory allocation routines for yajl which use malloc/realloc and
>> + * free
>> + */
>> +
>> +#include "yajl_alloc.h"
>> +#include <stdlib.h>
>> +
>> +static void * yajl_internal_malloc(void *ctx, unsigned int sz)
>> +{
>> +    return malloc(sz);
>> +}
>> +
>> +static void * yajl_internal_realloc(void *ctx, void * previous,
>> +                                    unsigned int sz)
>> +{
>> +    return realloc(previous, sz);
>> +}
>> +
>> +static void yajl_internal_free(void *ctx, void * ptr)
>> +{
>> +    free(ptr);
>> +}
>> +
>> +void yajl_set_default_alloc_funcs(yajl_alloc_funcs * yaf)
>> +{
>> +    yaf->malloc = yajl_internal_malloc;
>> +    yaf->free = yajl_internal_free;
>> +    yaf->realloc = yajl_internal_realloc;
>> +    yaf->ctx = NULL;
>> +}
>> +
>>
>> Added: couchdb/trunk/src/ejson/yajl/yajl_alloc.h
>> URL: http://svn.apache.org/viewvc/couchdb/trunk/src/ejson/yajl/yajl_alloc.h?rev=1088941&view=auto
>> ==============================================================================
>> --- couchdb/trunk/src/ejson/yajl/yajl_alloc.h (added)
>> +++ couchdb/trunk/src/ejson/yajl/yajl_alloc.h Tue Apr  5 09:42:41 2011
>> @@ -0,0 +1,50 @@
>> +/*
>> + * Copyright 2010, Lloyd Hilaiel.
>> + *
>> + * Redistribution and use in source and binary forms, with or without
>> + * modification, are permitted provided that the following conditions are
>> + * met:
>> + *
>> + *  1. Redistributions of source code must retain the above copyright
>> + *     notice, this list of conditions and the following disclaimer.
>> + *
>> + *  2. Redistributions in binary form must reproduce the above copyright
>> + *     notice, this list of conditions and the following disclaimer in
>> + *     the documentation and/or other materials provided with the
>> + *     distribution.
>> + *
>> + *  3. Neither the name of Lloyd Hilaiel nor the names of its
>> + *     contributors may be used to endorse or promote products derived
>> + *     from this software without specific prior written permission.
>> + *
>> + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
>> + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
>> + * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
>> + * DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT,
>> + * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
>> + * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
>> + * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
>> + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
>> + * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
>> + * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
>> + * POSSIBILITY OF SUCH DAMAGE.
>> + */
>> +
>> +/**
>> + * \file yajl_alloc.h
>> + * default memory allocation routines for yajl which use malloc/realloc and
>> + * free
>> + */
>> +
>> +#ifndef __YAJL_ALLOC_H__
>> +#define __YAJL_ALLOC_H__
>> +
>> +#include "yajl_common.h"
>> +
>> +#define YA_MALLOC(afs, sz) (afs)->malloc((afs)->ctx, (sz))
>> +#define YA_FREE(afs, ptr) (afs)->free((afs)->ctx, (ptr))
>> +#define YA_REALLOC(afs, ptr, sz) (afs)->realloc((afs)->ctx, (ptr), (sz))
>> +
>> +void yajl_set_default_alloc_funcs(yajl_alloc_funcs * yaf);
>> +
>> +#endif
>>
>>
>>
>

Mime
View raw message