Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (nike.apache.org: local policy)
Message-ID: <4AF64514.8090401@rogerbinns.com>
Date: Sat, 07 Nov 2009 20:12:04 -0800
From: Roger Binns <rogerb@rogerbinns.com>
User-Agent: Thunderbird 2.0.0.23 (X11/20090817)
MIME-Version: 1.0
To: user@couchdb.apache.org
Subject: Re: Silent corruption of large numbers
References: <4AF5FB9E.70300@rogerbinns.com>
 <e282921e0911071523n5633d6b8r6d1dbf5a2b43f684@mail.gmail.com>
 	<4AF622D5.9070708@rogerbinns.com>
 <e2111bbb0911071942o54914797h20e1471703e4fde@mail.gmail.com>
In-Reply-To: <e2111bbb0911071942o54914797h20e1471703e4fde@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Paul Davis wrote:
> RFC 4627 does mention this issue in passing in section 4 by saying "An
> implementation may set limits on the range of numbers." Generally I
> take this to mean that you're gonna have weird implementation
> dependent behavior when dealing with the limits of common number
> representations.

If there are limits then I'd expect them to be enforced in some way,
typically some sort of exception.  The last thing I would expect is for the
numbers to be silently corrupted.  If strings over 1,024 bytes were randomly
mutated would that be acceptable?

> As such, relying on the view engine to error out

In this case the limit is in the Javascript view engine.  The CouchDB server
doesn't have a problem.  If the view engine is Python then it won't have the
problem either.

> While not
> the best answer the only thing I can suggest would be to do as Adam
> says and store large values as strings and use a Bignum library in the
> places you need to manipulate such values.

In this case it is just my test suite that trips over the problem and my
language (Python) does bignums by default.  I'm not too bothered that there
is a failure but rather by the manner of the failure - silent mutation.

> Even if we told the view server to error on such values, what would
> that error look like? Would everyone be unable to pass a doc with a
> big num through the view server (depending on language)? Things get
> messy quick.

I'm old school.  I don't think this kind of mutation is acceptable.  A user
is sitting behind layers of user interface, CouchDB libraries, JSON
libraries, HTTP libraries and several other bits of glue.  Silently doing
this is bad - one day a user will end up with data corruption with serious
repercussions.

Using a string analogy what should a view server do if a string is passed in
that is larger than it wants to handle?  Is silent mutation or corruption ok?

Roger
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkr2RRIACgkQmOOfHg372QQjZACfdbGUvlT+t+ZzBQR9JKyAs4eD
th0AoNAJk/XAQi2qSpA9QkBrHWnU4NFH
=wHXn
-----END PGP SIGNATURE-----