From user-return-7481-apmail-couchdb-user-archive=couchdb.apache.org@couchdb.apache.org Tue Nov 10 07:36:37 2009 Return-Path: Delivered-To: apmail-couchdb-user-archive@www.apache.org Received: (qmail 77547 invoked from network); 10 Nov 2009 07:36:37 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 10 Nov 2009 07:36:37 -0000 Received: (qmail 32420 invoked by uid 500); 10 Nov 2009 07:36:36 -0000 Delivered-To: apmail-couchdb-user-archive@couchdb.apache.org Received: (qmail 32350 invoked by uid 500); 10 Nov 2009 07:36:35 -0000 Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@couchdb.apache.org Delivered-To: mailing list user@couchdb.apache.org Received: (qmail 32340 invoked by uid 99); 10 Nov 2009 07:36:35 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 10 Nov 2009 07:36:35 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [63.249.119.34] (HELO server.rogerbinns.com) (63.249.119.34) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 10 Nov 2009 07:36:25 +0000 Received: from [192.168.1.53] (unknown [192.168.1.53]) by server.rogerbinns.com (Postfix) with ESMTPSA id B51FD21146B for ; Mon, 9 Nov 2009 23:36:03 -0800 (PST) Message-ID: <4AF917E4.4000201@rogerbinns.com> Date: Mon, 09 Nov 2009 23:36:04 -0800 From: Roger Binns User-Agent: Thunderbird 2.0.0.23 (X11/20090817) MIME-Version: 1.0 To: user@couchdb.apache.org Subject: Re: Silent corruption of large numbers References: <4AF622D5.9070708@rogerbinns.com> <20091109094640.GA8456@uk.tiscali.com> <4AF8797B.6090206@rogerbinns.com> In-Reply-To: X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Paul Davis wrote: > The only answer that sounds sensible to me is to code defensively and > make your own guarantees on data passing. The Couch world is only > going to expand, so data will come and go from a huge multitude of > sources and through any number of intermediaries. The last phrase is what scares me. My component being defensive won't protect the ultimate end user from the other parts. > Or in a more specific case, I don't even know how I would get the JS > view engine to raise an error on this condition. Except for taking the > raw string, deserializing, reserializing, and comparing, and even that > is obviously prone to lots of other oddities of escaping and so on and > such forth. I'm advocating work for other people here but perhaps some sort of test/qualification suite for view servers would do the trick. It would work for me having a page listing the CouchDB server itself and then view servers and what their limits and quirks are as established by the aforementioned test/qualification suite. For example the number of significant digits for floating point, integer ranges, string sizes etc can be listed. At a guess there are also limits on string sizes, Unicode capabilities (codepoints above 0xffff), surrogate pair handling), JSON items (eg number of items in a JSON list, keys in an object etc). I am guessing there are various 32 bit limits around even on 64 bit platforms. At some point I'll try storing 5GB strings or lists with more than 4 billion items and see what happens :-) > In trunk the computation and index writing are split to take up to two > cores. Cool. Currently what I see is beam.smp and couchjs in total taking up 100% of a cpu. Using beam.smp is taking up the most but sometimes it is couchjs at 53% and beam.smp at 47%. Roger -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkr5F+IACgkQmOOfHg372QTDpQCePRuT5uP7/jX23U5dh8VcmI/v AgQAn2foQMt6jHGgU3hvQ7FZt14vfAgb =Oujm -----END PGP SIGNATURE-----