couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Davis <paul.joseph.da...@gmail.com>
Subject Re: Futon Test Suite
Date Tue, 09 Aug 2011 15:19:11 GMT
On Tue, Aug 9, 2011 at 9:38 AM, Adam Kocoloski <kocolosk@apache.org> wrote:
> On Aug 9, 2011, at 4:48 AM, Paul Davis wrote:
>
>> On Tue, Aug 9, 2011 at 2:40 AM, Robert Dionne
>> <dionne@dionne-associates.com> wrote:
>>>>
>>>>
>>>> Also, I've been thinking more and more about beefing up the JavaScript
>>>> test suite runner and moving more of our browser tests over to
>>>> dedicated code in those tests. If anyone's interested in hacking on
>>>> some C and JavaScript against an HTTP API, let me know.
>>>
>>>
>>> Paul,
>>>
>>>  Jan and I talked about this a few times and I started a branch[1] along that
idea. So far all I did was make a copy of the then current Futon tests into
>>> test/javascript/test  and started looking at the small handful that fail.
>>>
>>>   The browser tests are great (any test is good) but they have too many browser
 dependent quirks, or at least I assume that because of the pleasant surprise
>>> one gets when they all run. So I think the goal of these runner tests would be
some sort of official HTTP API suite that's part of "make check". Would you agree?
>>> If so I'm happy to take this on.
>>>
>>>   Also I've found eunit to be helpful in BigCouch and wondering how hard it
would be to support eunit in couchdb. Having tests in the modules is very good for not
>>> only testing but also to help with reading and understanding what the code does.
>>>
>>> Bob
>>>
>>>
>>> [1] https://github.com/bdionne/couchdb/tree/cli-tests
>>>
>>>
>>
>> Bob,
>>
>> Exactly what I was thinking. By having our test suite have as little
>> code between the socket and the the test as possible we can ensure
>> that the tests are testing CouchDB and not some random change in the
>> behavior of our favorite web browser. I would definitely expect these
>> tests to be part of make check and hence part of the release
>> procedure.
>>
>> My current half formed thoughts are to basically split our test suite
>> into two halves. Tests that are in Erlang and are testing internals,
>> and tests that go through the HTTP interface. I love me some Erlang,
>> but I've not been think of an elegant way to make it easy to run lots
>> of HTTP tests.
>>
>> As to eunit, I'm not sure. I'm really not a huge fan of it, especially
>> mixing implementation and test code. I know rebar can separate them,
>> so its at least possible to get around that. I'd like to have a
>> unified environment for Erlang tests though. And TAP at seems like
>> it'd be easier to interface with non-Erlang tooling if we ever get
>> around to build matrices and what not. But I'm not opposed to it on
>> religious grounds if that's what people want to contribute.
>
> OTP ships with eunit_surefire[1] so interfacing with general test infrastructures isn't
really an issue.  I had to dig a little deeper to find a decent TAP consumer for Jenkins
and ended up executing the tests from a Perl script using TAP::Harness::JUnit.  I'll grant
that the TAP output is easier for a human to digest than the eunit output.
>
> I imagine testing the private module functions is a topic for a good flamewar.  I find
it useful, but maybe that's a sign I've got too much logic in a particular module.  Either
way,  a viable alternative to writing unit tests that execute in the browser is something
this project really needs.  Cheers,
>
> Adam
>
> [1]: http://www.erlang.org/doc/man/eunit_surefire.html

Way to totally destroy on of my last hopes of not using eunit. Thanks for that.

Also, yes. I've finally become irritated enough with clearing the
browser cache between every test that I feel its time to do something
productive about it.

Mime
View raw message