From couchdb-dev-return-704-apmail-incubator-couchdb-dev-archive=incubator.apache.org@incubator.apache.org Mon Jul 07 17:26:06 2008 Return-Path: Delivered-To: apmail-incubator-couchdb-dev-archive@locus.apache.org Received: (qmail 56219 invoked from network); 7 Jul 2008 17:26:06 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 7 Jul 2008 17:26:06 -0000 Received: (qmail 34528 invoked by uid 500); 7 Jul 2008 17:26:06 -0000 Delivered-To: apmail-incubator-couchdb-dev-archive@incubator.apache.org Received: (qmail 34346 invoked by uid 500); 7 Jul 2008 17:26:06 -0000 Mailing-List: contact couchdb-dev-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: couchdb-dev@incubator.apache.org Delivered-To: mailing list couchdb-dev@incubator.apache.org Received: (qmail 34335 invoked by uid 99); 7 Jul 2008 17:26:06 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 07 Jul 2008 10:26:06 -0700 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of damienkatz@gmail.com designates 64.233.170.189 as permitted sender) Received: from [64.233.170.189] (HELO rn-out-0910.google.com) (64.233.170.189) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 07 Jul 2008 17:25:11 +0000 Received: by rn-out-0910.google.com with SMTP id m7so442534rnd.3 for ; Mon, 07 Jul 2008 10:25:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:from:to :in-reply-to:content-type:content-transfer-encoding:mime-version :subject:date:references:x-mailer; bh=pur99Tl4BBIBi41gpCJ0wmN8eDnFAK+dYfsMs1k2kDA=; b=R5JfhpA5ys3ViWBLDVmprUu6ourkqXm6z/FQrWEXVfS7SYhlhffqAnhuRHpX0E7hRf jWkhyAim7mob6yyfMoqOY6W4lhVewzEw3ak5cLDTDFhAdN8j2agkJHEXWpangr/hlbJU Z5AGpHzHPoBrDw3Sj51BNaJ2ZPMeXSpErpl0Y= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:from:to:in-reply-to:content-type :content-transfer-encoding:mime-version:subject:date:references :x-mailer; b=pqOQjY7URKaJogtzOgdrrBif91wYtCH2QYVlv8aoX1+vV4+35U583mC0PS6rOn84jG fongmqZ+g4IDLfKeJ1JUN1ftehZ6UmZXllJdAvjnXvfEcaTEo15NmFpTuBD5xyRFdTc5 gcEiiuf1/gy+vx5Tr8RrbcqjCixWtLcBZisuc= Received: by 10.150.212.12 with SMTP id k12mr8512513ybg.228.1215451531502; Mon, 07 Jul 2008 10:25:31 -0700 (PDT) Received: from ?192.168.1.199? ( [96.33.90.152]) by mx.google.com with ESMTPS id z80sm14642420pyg.31.2008.07.07.10.25.29 (version=TLSv1/SSLv3 cipher=RC4-MD5); Mon, 07 Jul 2008 10:25:30 -0700 (PDT) Message-Id: From: Damien Katz To: couchdb-dev@incubator.apache.org In-Reply-To: <5C63EDBB-B530-4D08-9CA1-3C484E74F1C7@apache.org> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v926) Subject: Re: cjson.erl Date: Mon, 7 Jul 2008 13:25:28 -0400 References: <561eba880807070151g18e6939er241449695c4acbe4@mail.gmail.com> <5C63EDBB-B530-4D08-9CA1-3C484E74F1C7@apache.org> X-Mailer: Apple Mail (2.926) X-Virus-Checked: Checked by ClamAV on apache.org So the sad history of cjson.erl I started with the erlang json library I found on the json.org website (which now appears to be a dead link), and used that for a while. For reasons I cannot remember (bugs or performance), I switched to using the mochiweb json library. However, it used slightly different conventions for using Erlang terms to represent the Json. For one thing, objects were {struct, [...]}, while the json.org library used {obj, [...]}. I think there was one other thing, but I can't remember now. Anyway, rather than change all my code to use the new convention, I change the mochi library it to use the json.org conventions and changed the name to cjson.erl (for reason I again cannot remember). Some of the comments in the library are likely wrong because of this. Now switching libraries is easy, but switching the Erlang respresentation of json objects is not. However I'd be glad to switch over CouchDB to using a different Erlang representation of json, if there is a "blessed" Erlang format. Otherwise, I'll need practical reasons for doing so. Performance is one such reason. One thing I'm no happy about is the idea of representing strings using binaries. From a code asthetics point of view, it uglifies the source dramatically, but I think it might also cause lots of extra conversions between binary strings and normal list strings used in most Erlang libraries and APIs. If the memory and performance improvements will have to be big to make up for the extra complexities in the source. -Damien On Jul 7, 2008, at 12:24 PM, Jan Lehnardt wrote: > Heya, > Joe Armstrong tries to get the Erlang community to agree > on a single JSON library that fits everybody's needs. The > biggest players here (according to Joe I guess) are > MochiMedia and ourselves. > > Hence the dialogue I quote below: > > Begin forwarded message: > >> From: "Joe Armstrong" >> Date: July 7, 2008 10:51:07 AM GMT+02:00 >> To: "Jan Lehnardt" >> Cc: "Bob Ippolito" >> Subject: cjson.erl >> >> Hi Jan, >> >> [CC'd to Bob Ippolito (Glad to see the facebook stuff taking off - >> great work :-)) ] >> >> I've been staring at cjson.erl ... >> >> The comments say it's derived from mochijson.erl. >> >> In the mochiweb there are two json representations >> mochijson2.erl and mochijson.erl >> >> I think the "2" is the better one :-) >> >> I think it would be a good idea if you could come to some agreement >> with the mochiweb people as to the best representation of >> JSON terms in ERlang and both go out with a single library. >> >> cjson.erl lacks a type declaration in the documentation - which it >> needs >> (reading the code is hopeless) >> >> mochijson2.erl has this type declaration >> >> %% @type json_string() = atom | binary() >> %% @type json_number() = integer() | float() >> %% @type json_array() = [json_term()] >> %% @type json_object() = {struct, [{json_string(), json_term()}]} >> %% @type json_term() = json_string() | json_number() | json_array() | >> %% json_object() >> >> I'm not sure about the additional "struct" tag - nor the additional >> atom tag in json_string >> >> How about ... >> >> @type json_object = {[json_tag::binary(), json_term()]} >> @type json_string() = binary() >> >> this makes the erlang term map to JSON in an unambigous manner and >> the compiler should be able to generate faster code, since >> >> unpack(Json) when is_binary(J) -> ... >> >> will only have disjoint branches. >> >> I think that: >> >> lists should *only* be used for json_arrays >> binary should *only* be used for json_strings >> json objs should be *only* be tuples (of pairs) {{Tag,Val}, >> {Tag,Val},...} >> (possibly {Tag1,Val1,Tag2,Val2,....} might be better???) >> >> I think it would be a good idea to isolate this problem - agree >> (having done some >> measurements, on the fastest and *prettiest* way to do this) - >> jointly change >> your code bases (at the same time) and then tell the world - then >> issue ONE >> library. >> >> Just for fun I've downloaded the wikipedia using the ideas in >> >> http://users.softlab.ece.ntua.gr/~ttsiod/buildWikipediaOffline.html >> >> (I want to converts the XML representation of the wikipedia into JSON >> and inject it into coutchDB >> and serve it up with mochiweb - I need to write a rendering engine to >> convert wiki markup to HTML >> (this is said to be tricky since there is no spec :-) >> >> This should be a good test of coutchDB and mochiweb) >> >> Cheers >> >> /Joe Armstrong > > > And Bob's reply: > >> From: "Bob Ippolito" >> Date: July 7, 2008 6:12:32 PM GMT+02:00 >> To: "Joe Armstrong" >> Cc: "Jan Lehnardt" >> Subject: Re: cjson.erl >> >> {struct, ...} is what the library that ships with Yaws does, which is >> why I used that. Using just {[{Key, Value}]} looks fine to me also >> and >> should be do-able without breaking compatibility immediately. >> >> The reason atoms are accepted is only for encoding purposes, not for >> decoding. There is an unambiguous format from JSON -> Erlang but for >> Erlang -> JSON some conveniences are allowed for practical reasons. >> >> I'm fine with the {struct, ...} -> {...} change that Joe proposed >> because I can do that in a backwards compatible way. >> >> -bob > > What is our take on this? :) Damien? > > I'll forward our discussions back to Joe and Bob (in case they don't > read this list). > > Cheers > Jan >