Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
Received-SPF: pass (athena.apache.org: local policy)
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Apple Message framework v1084)
Subject: Re: futon tests (Was: Re: [VOTE] Apache CouchDB 1.1.1 Release)
From: "J. Lee Coltrane" <lee@projectmastermind.com>
In-Reply-To: 
 <CALpjNy=-wjJRMVn-V6E4TsfoG4Gu=jQpJ8rm4zJxJZ0eKvxR9A@mail.gmail.com>
Date: Wed, 19 Oct 2011 23:38:26 -0400
Content-Transfer-Encoding: 7bit
Message-Id: <00EFB465-11C2-4023-ACEA-F80D3E3E5CB2@projectmastermind.com>
References: 
 <CALpjNyk1C28d4xTVrpGyObKzYQeKKysDLAeFrn4y-cMR43GawQ@mail.gmail.com>
 <CAJ_m3YCeDrBXysNNWuPqzT0Qik=XDR3dnzOADRXT_VkJu+G-zA@mail.gmail.com>
 <CALpjNy=-wjJRMVn-V6E4TsfoG4Gu=jQpJ8rm4zJxJZ0eKvxR9A@mail.gmail.com>
To: dev@couchdb.apache.org


> For what it's worth, a CLI based test system is what I was imagining
> as well. Take Futon out of the mix and test CouchDB.

IMO, If CouchDB is intended to be a server that can be accessed from
the browser directly, then there should continue to be some kind of 
browser-based test suite that would serve to confirm this capability.  


I have been looking closely at the Futon tests in 1.1.0 for the last
several days, with the idea that I might begin to clean them up a bit
as time permits.

I have found that, while some of these test failures are totally bogus, 
*some* of them actually do stem from real issues -- minor 
incompatibilities between CouchDB's http interface, and the internal 
mechanisms of modern browsers (XHR, caching, etc).

These are problems that we're not going to catch with a stateless, 
cache-less http client running on the CLI.  (I can provide examples)

These issues have the potential to cause real problems for 
developers of real browser-based apps "in the wild".  That means, 
there's valuable info to be gathered from the browser tests, Iff we 
can clean them up, and make them behave consistently; so that 
when they fail or succeed, we can actually trust the results.


After digging around a good bit, I can see no reason why the existing 
tests couldn't be cleaned up and made to work correctly in all current 
versions of major browsers.  I also see no reason why the same tests 
couldn't be used successfully from the CLI and `make check` as well.

I do see significant benefits to using the same javascript test code in
all environments we test.

-Lee
(irc: coltr)