Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
Content-Type: text/plain; charset=iso-8859-1
Mime-Version: 1.0 (Apple Message framework v1257)
Subject: Re: [VOTE] Apache CouchDB 1.2.0 release, first round
From: Jan Lehnardt <jan@apache.org>
In-Reply-To: 
 <CAJ_m3YDhYtzQOw4iYcgu54H86-jvPJYE=J_VaVnDYE_eEFH6DQ@mail.gmail.com>
Date: Mon, 13 Feb 2012 11:54:19 +0100
Content-Transfer-Encoding: quoted-printable
Message-Id: <2F10FA8E-60F7-491A-8F6F-47CF0F64D8FC@apache.org>
References: 
 <CA+Y+444JKqhZvfZduqMmqaR1nCZU9Uvts_W9QCjxSfsnFdwmfA@mail.gmail.com>
 <CAN-3CBJDzfqgWc+Ouvdue8Xs+WO1b5ZCQrYBTZHRb3TPRGVkEw@mail.gmail.com>
 <CA+Y+444LOQ6=jYSKcmFZLH55xAOVfUd_m5V0xGuCeoTRLHmLag@mail.gmail.com>
 <CAN-3CBJ+tad3A5cjwKYTV9TkK=k5P35=nkaWfkNmhagZ8Cdnkw@mail.gmail.com>
 <CAAL6JQja5eXOkuqnWr2eT+kKHfufHyC2pFPp2TRiwjLjcJz-gA@mail.gmail.com>
 <CAAL6JQjhNWJTrX6s9xni=11-PP2V-3nmdA+1VDMjieLxarJWNQ@mail.gmail.com>
 <CAN-3CBJ0LGTc3gXGQc9Fa+2LKEqo4Q2fkOYvo1kNdGDSUu9hGQ@mail.gmail.com>
 <CAJNb-9pE6ZdiJ3Za6_uRXRWbH05p88QjUg5wxN6Vuc22nYt=CQ@mail.gmail.com>
 <CABvT1DGqPdKvytCTiNknwhuB7EotTbgHUR+cx2y0VEV0NUt-Cw@mail.gmail.com>
 <CAJ_m3YDhYtzQOw4iYcgu54H86-jvPJYE=J_VaVnDYE_eEFH6DQ@mail.gmail.com>
To: dev@couchdb.apache.org

Excellent analysis Paul.

I'd say we go with the patch for 1.2.0 and beyond.

Cheers
Jan
--=20

On Feb 13, 2012, at 07:55 , Paul Davis wrote:

> So yeah. Numbers are hard.
>=20
> Firstly, anyone that mentioned RFC 4627 or JavaScript behavior is
> walking down a path entirely orthogonal to the issue at hand. Jason
> almost had it when he talked about them being different but then he
> went off on some weird tangent and lost me.
>=20
> In a nutshell, the issue is this:
>=20
> CPU's work with bits. Humans (and JSON sorta) work with numbers as a
> string of numerals with some punctuation. This is a lossy conversion.
>=20
> So, back to the details.
>=20
> COUCHDB-1407 reports that ejson now encodes the value "1.0" as "1".
> While we can wax philosophically about this, the bottom line is that
> this breaks a level of equality. Specifically:
>=20
> 1> ejson:decode(ejson:encode(1.0)) =3D:=3D 1.0.
> false
>=20
> This is precisely because the %g formatting used underneath removes
> trailing zeros and decimal points.
>=20
> On the face of it, this is bad. And I agree. There's a simple enough
> fix (and its not what Bob Newson suggested, but I'm going to leave him
> hanging for a bit).
>=20
> But, before we get all crazy, we should contemplate a few other fun =
cases:
>=20
> 1. Both mochijson2 and ejson change some number representations
>=20
> 5> mochijson2:encode(mochijson2:decode("1E1")) =3D:=3D "1E1".
> false
> 6> ejson:encode(ejson:decode("1E1")) =3D:=3D "1E1".
> false
>=20
> 2. Both mochijson2 and ejson turn numbers with exponents into IEEE-754
> internally
>=20
> 7> ejson:decode("1E1") =3D:=3D 10.
> false
> 8> mochijson2:decode("1E1") =3D:=3D 10.
> false
>=20
> 3. Others but I'm tired from staring at math.
>=20
> Basically, the end result is that we can match mochijson2's decoder
> damn near identically (At least, I know of no known differences in
> decoding in Jiffy). But now we get to the hard part.
>=20
> Mochijson2 does some fancy ass magic for encoding IEEE-754 values. And
> when I say fancy, I mean, implements an algorithm published in some
> random paper from 1996 based on the paper's author's Scheme
> implementation. I spent about twelve hours today trying to duplicate
> before I realized that it depends on having an integral type that can
> represent values with more than 64 bits (which made me sad).
>=20
> EIther way, this is dark voodoo. Anyone that's interested can checkout
> mochinum:digits/1 and the supporting functions for some mind bending
> looks into IEEE-754 representations.
>=20
> Anyway, bottom line is that 1.0 should be encoded as "1.0". The fix is
> simply to just check for a decimal point and append one if its not
> there. This is what Yajl does and Python appears to behave similarly.
> The patch for Jiffy is at [1] and shows the general idea.
>=20
> Also, for those still holding on to why %f is not a valid fix, the
> reason is the same as why %g is wrong (and why it needs to be %0.20g.
> printf and friends by default will round to the sixth decimal places.
> So, 0.123456789 would get encoded as "0.123457" which loses precision.
>=20
> Also, with that patch for Jiffy we never lose precision but the
> eyesore is that we encoded 0.1 as "0.10000000000888" (Roughly). Some
> people find that offensive but I don't really care enough to learn
> arbitrary precision math routines so people can have slightly prettier
> JSON. And I say that after having spent all day trying to make it
> work.
>=20
> So, yeah. Fix is simple enough.
>=20
> Also, food for thought: A JSON parser/serializer pair that converts
> all numbers to 42 is technically compliant with the JSON spec.
>=20
> [1] =
https://github.com/davisp/jiffy/commit/5042cc946008ee413cc66b9b0addcf33ecd=
2fd93
>=20
> On Sat, Feb 11, 2012 at 8:32 AM, Robert Newson <rnewson@apache.org> =
wrote:
>> I'd like some opinions on whether COUCHDB-1407 constitutes a release
>> blocking issue. Yes, I understand that the JSON spec is very weak on
>> numbers, blah blah boo splat. Is this because of the switch to ejson?
>> Is jiffy more compatible on this score?
>>=20
>> For my part, I'm close to considering it a release-blocking
>> regression. At the very least this change should be included at
>> =
http://wiki.apache.org/couchdb/Breaking_changes#Changes_Between_1.1.0_and_=
1.2.0
>> but I'd rather it was fixed.
>>=20
>> B.
>>=20
>> On 11 February 2012 10:44, Benoit Chesneau <bchesneau@gmail.com> =
wrote:
>>> On Sat, Feb 11, 2012 at 4:00 AM, Jason Smith <jhs@iriscouch.com> =
wrote:
>>>> On Sat, Feb 11, 2012 at 3:06 AM, Randall Leeds =
<randall.leeds@gmail.com> wrote:
>>>>> On Feb 9, 2012 6:09 PM, "Randall Leeds" <randall.leeds@gmail.com> =
wrote:
>>>>>>=20
>>>>>> On Thu, Feb 9, 2012 at 17:48, Jason Smith <jhs@iriscouch.com> =
wrote:
>>>>>>> Hi, Noah. When I saw it hit Git, I realized it was a breaking =
change,
>>>>>>> and I asked around. If memory serves, Randall happened to be on =
at the
>>>>>>> time and he asked me the same question you just did. I said I =
never
>>>>>>> saw an RFC email and that's when he realized it was not done =
publicly.
>>>>>>=20
>>>>>> I was aware the entire time, but I think the motivation is sound =
and
>>>>>> it needed to be done. A couple committers spoke up to say we =
didn't
>>>>>> think it was sensitive enough to warrant the private discussion =
but
>>>>>> ultimately there was broad consensus on the implementation and =
the
>>>>>> change itself. One of those (let us all celebrate) extremely rare
>>>>>> times where there wasn't opportunity for broad community input.
>>>>>>=20
>>>>>> Creating a view on _users that pulls the relevant parts of a user
>>>>>> document out is the way forward for public profiles, I think.
>>>>>> If someone would write a blog post showing how that works it'd be
>>>>>> great. In retrospect this would have been a great thing to do =
weeks
>>>>>> ago. Lesson learned.
>>>>>=20
>>>>> Just to be clear I don't want to dismiss your concerns. If you =
believe this
>>>>> needs a config option rather than just documentation now is a good =
time to
>>>>> speak up loudly since the vote was aborted.
>>>>=20
>>>> Thanks. I am concerned. To me, the change is noteworthy but not a =
showstopper.
>>>>=20
>>>> I tested your suggestion, however I do not think it is possible.
>>>> Non-admins cannot access a view.
>>>>=20
>>>> $ curlp http://admin:admin@localhost:5984/_users/_design/public -d
>>>> '{"views":{"all":{"map":"function(doc) { emit(doc._id, doc) }"}}}'
>>>> =
{"ok":true,"id":"_design/public","rev":"1-f605d1ea7825645132f54a91a76a1ddc=
"}
>>>>=20
>>>> $ curl -i =
http://user:user@localhost:5984/_users/_design/public/_view/all
>>>> HTTP/1.1 403 Forbidden
>>>> Server: CouchDB/1.2.0 (Erlang OTP/R15B)
>>>> Date: Sat, 11 Feb 2012 02:57:43 GMT
>>>> Content-Type: text/plain; charset=3Dutf-8
>>>> Content-Length: 102
>>>> Cache-Control: must-revalidate
>>>>=20
>>>> {"error":"forbidden","reason":"Only admins can access design =
document
>>>> actions for system databases."}
>>>>=20
>>> Yes that's by design.
>>>=20
>>> - beno=EEt