couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jim Klo <jim....@sri.com>
Subject Re: Fixed precision of floating point number not respected in views
Date Wed, 20 Feb 2013 15:07:10 GMT


On Feb 20, 2013, at 2:34 AM, "Robert Newson" <rnewson@apache.org> wrote:

> The transport protocol or storage format is not the issue here,
> though. If you're simply displaying this value, then the suggestion to
> use strings should not be controversial. The only reason to care this
> much about numeric representation is so you can do math.

This much I know... It didn't need rehashing for the infinite time.. :)

It was only unclear what places FP values become "lossy" (on insert, indexing, query, etc).


> As several
> folks have noted, if you're using floating point for handling money
> values, you are in a state of sin already. For many other
> applications, 64-bit fp math is pretty good. Your problem domain
> should tell you whether you require specific behavior from your
> numeric data types (like specific rounding algorithms, or arbitrary
> precision).
> 

All correct. I was considering the use of protobufs to improve the accuracy of the math as
it sounded like the roundtrip from storage introduced error - and maybe this is where I'm
missing something; which is CouchDB specific.  If trying to do precision math within MapReduce,
show, list, etc... Is the precision issue occur between what is returned/emitted from JS (which
I presume gets handed to Erlang and then written to output)? Of which the only thing I could
think that could be used to eliminate would be to store in a string and use Erlang views and
such unless Erlang is what's at fault for dropping accuracy - assuming precision math is even
possible in Erlang. 

> 
> I think we've said this every way it can be said now.
> 

We could start in foreign languages... 
:-)

> B.
> 
> 
> 
> On 20 February 2013 08:37, Jim Klo <jim.klo@sri.com> wrote:
>> I'm not sure if this is a terrible idea or not... (As I've never tried it, but got
the idea while following this thread) But how about encoding with protocol buffers when storing
into couch... There are a few protocol buffers JS libraries that could be used to marshall
values within MapReduce, list, show, and other functions... but then you possibly run up against
spidermonkeys precision issues, at least there would be no Erlang to Spidermonkey round trip
issues. With the amount of data you seem to be concerned about, it might even help with space,
especially if you made list/show functions that streamed in protobufs. Just not sure how viable
or practical it would be.
>> 
>> I know I've run into this issue before and have conceded to store FP as an array
of 2 int values (mantissa and exponent) as well as encoded with strings. However I've not
needed to do any Maths within CouchDB, just pure storage.
>> 
>> Sent from my iPhone
>> 
>> On Feb 20, 2013, at 12:03 AM, "Benoit Chesneau" <bchesneau@gmail.com> wrote:
>> 
>>> On Tue, Feb 19, 2013 at 6:33 PM, Jens Alfke <jens@couchbase.com> wrote:
>>> 
>>>> As for alternative data formats, there are a lot of them around, and I don’t
see that EDN would have an advantage over any of the others that are better known and more
widely adopted, like YAML. Changing away from JSON would be a compatibility nightmare, anyway
(speaking as the author of a Couch-compatible replication engine.)
>>> 
>>> 
>>> edn has the advantage to provide you a way to describe the number by
>>> annotating it somehow. So it doesn't  solve the "maths" problem but
>>> would solve the representation or at least make it a little better
>>> when for example you expect a real with 2 decimals  you will get it
>>> instead of having X non expected decimals.
>>> 
>>> Going to a new format would be for sure a challenge but not that big
>>> if apache couchdb and compatible product respect the content-type
>>> header. And it will be a user choice.
>>> 
>>> - benoît

Mime
View raw message