incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Davis <paul.joseph.da...@gmail.com>
Subject Re: Fixed precision of floating point number not respected in views
Date Tue, 19 Feb 2013 11:13:05 GMT
On Tue, Feb 19, 2013 at 4:40 AM, Luca Morandini <lmorandini@ieee.org> wrote:
> On 02/19/2013 08:09 PM, Robert Newson wrote:
>>
>> "I have stored fixed precision numbers in a database". You haven't,
>
>
> Well, actually, I have.
>
> When I see them in Futon,  I can see they are stored as I had wrritten them;
> it is only when I use JavaScript (or access a document through the Document
> API) that those numbers are interpreted in an inconvenient manner.
>

This is because you're viewing these numbers through the lens of your
browser which hides some details. The document API is the ground truth
for what data CouchDB is returning.

> That said, I was wondering whether a different view engine may solve this
> issue.
>

No, it won't help. Even if the view engine serialized into a minimal
representation the Erlang decoder/encoder pair still stores values in
an internal format. The encoding from this format is what gives you
the undesirable trailing digits.

>
>
>> If you want fixed precision, you'll need to store your numbers in
>> strings and manipulate them that way too. A quick google in the past
>> has shown a few "bignum" libraries for Javascript.
>
>
> No, storing as string won't do, since I have little contro0l over the
> content of documents to be stored... I could write an handler function to
> transform every number into a string, but this would sap performance.
>

The precise issue here is that JSON decoders almost unanimously
deserialize numbers that include a decimal point or exponent into a
native floating point format internally. This generally means either a
(32bit) float or (64bit) double. This means that there is a precise,
finite set of numbers that can be represented in a byte identical
round trip through an encoder/decoder pair.

What Bob is suggesting is that if you want to maintain precise
numerical fidelity then passing numbers through such a conversion is
untenable. If you need absolute fidelity in numerical computing the
only real answer is to ensure that your numbers only pass through
libraries that are specifically designed with such constraints in
mind.

In general, when using JSON this means that you need to ensure your
numbers are represented as strings and use a custom library on either
end, or you need to restrict your use of JSON decoder/encoder pairs to
things that make such assertions. (I know of exactly zero libraries
that makes this assertion for arbitrary numbers, even Python which is
the best round-trip-fidelity library I'm aware of).

>
> Regards,
>
> Luca Morandini
> Data Architect - AURIN project
> Department of Computing and Information Systems
> University of Melbourne
> Tel. +61 03 903 58 380
> Skype: lmorandini
>

Mime
View raw message