Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
Received-SPF: pass (nike.apache.org: domain of paul.joseph.davis@gmail.com
 designates 209.85.220.180 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <434F9286-FBB6-419E-A585-4E091C10F5B8@telavox.se>
References: 
 <CAJ_m3YDr_G34C5SK=61hbkNWCP52=F46odGHn7wam7eU3cBktQ@mail.gmail.com>
 <EB43EB02-71C0-40C8-9852-1B573DB3BB23@dionne-associates.com>
 <CAJ_m3YD56cSuuhD_3Vv7SVsJrXw-_jyk8CUWms22FHLOZUjBLg@mail.gmail.com>
 <434F9286-FBB6-419E-A585-4E091C10F5B8@telavox.se>
From: Paul Davis <paul.joseph.davis@gmail.com>
Date: Tue, 9 Aug 2011 10:17:32 -0500
Message-ID: 
 <CAJ_m3YDGB4oEAhU7xNHnhRQg=KxnvH9ogLdUCRiihSBLf9KjNg@mail.gmail.com>
Subject: Re: Futon Test Suite
To: dev@couchdb.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On Tue, Aug 9, 2011 at 4:11 AM, Jens Rantil <jens.rantil@telavox.se> wrote:
> Hi,
>
> Does JS engine come bundled with HTTP request implementation or is that b=
rowser specific? If they do, you could execute the browser tests by running=
 them through a JavaScript engine directly? That way you can preserve the J=
avaScript code, still run the tests and skip your favorite browser.
>
> My five cents,
>
> /Jens
>
> Sent from my cellphone
>

This is precisely how our current JS CLI test runner works. There's a
directory ./test/javascript/ in the tarballs and in SVN that contains
the code to turn CouchJS into a minimal JS environment with a fake XHR
class. The idea currently floating in my head involves abandoning the
"pretend to be a browser" approach and is more focused on creating a
useful HTTP testing environment in JS and then refactoring our test
code to use that instead.

> On 9 aug 2011, at 10:49, "Paul Davis" <paul.joseph.davis@gmail.com> wrote=
:
>
>> On Tue, Aug 9, 2011 at 2:40 AM, Robert Dionne
>> <dionne@dionne-associates.com> wrote:
>>>>
>>>>
>>>> Also, I've been thinking more and more about beefing up the JavaScript
>>>> test suite runner and moving more of our browser tests over to
>>>> dedicated code in those tests. If anyone's interested in hacking on
>>>> some C and JavaScript against an HTTP API, let me know.
>>>
>>>
>>> Paul,
>>>
>>> =A0Jan and I talked about this a few times and I started a branch[1] al=
ong that idea. So far all I did was make a copy of the then current Futon t=
ests into
>>> test/javascript/test =A0and started looking at the small handful that f=
ail.
>>>
>>> =A0 The browser tests are great (any test is good) but they have too ma=
ny browser =A0dependent quirks, or at least I assume that because of the pl=
easant surprise
>>> one gets when they all run. So I think the goal of these runner tests w=
ould be some sort of official HTTP API suite that's part of "make check". W=
ould you agree?
>>> If so I'm happy to take this on.
>>>
>>> =A0 Also I've found eunit to be helpful in BigCouch and wondering how h=
ard it would be to support eunit in couchdb. Having tests in the modules is=
 very good for not
>>> only testing but also to help with reading and understanding what the c=
ode does.
>>>
>>> Bob
>>>
>>>
>>> [1] https://github.com/bdionne/couchdb/tree/cli-tests
>>>
>>>
>>
>> Bob,
>>
>> Exactly what I was thinking. By having our test suite have as little
>> code between the socket and the the test as possible we can ensure
>> that the tests are testing CouchDB and not some random change in the
>> behavior of our favorite web browser. I would definitely expect these
>> tests to be part of make check and hence part of the release
>> procedure.
>>
>> My current half formed thoughts are to basically split our test suite
>> into two halves. Tests that are in Erlang and are testing internals,
>> and tests that go through the HTTP interface. I love me some Erlang,
>> but I've not been think of an elegant way to make it easy to run lots
>> of HTTP tests.
>>
>> As to eunit, I'm not sure. I'm really not a huge fan of it, especially
>> mixing implementation and test code. I know rebar can separate them,
>> so its at least possible to get around that. I'd like to have a
>> unified environment for Erlang tests though. And TAP at seems like
>> it'd be easier to interface with non-Erlang tooling if we ever get
>> around to build matrices and what not. But I'm not opposed to it on
>> religious grounds if that's what people want to contribute.
>