couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Kocoloski <kocol...@apache.org>
Subject Re: Futon Test Suite
Date Tue, 09 Aug 2011 14:38:09 GMT
On Aug 9, 2011, at 4:48 AM, Paul Davis wrote:

> On Tue, Aug 9, 2011 at 2:40 AM, Robert Dionne
> <dionne@dionne-associates.com> wrote:
>>> 
>>> 
>>> Also, I've been thinking more and more about beefing up the JavaScript
>>> test suite runner and moving more of our browser tests over to
>>> dedicated code in those tests. If anyone's interested in hacking on
>>> some C and JavaScript against an HTTP API, let me know.
>> 
>> 
>> Paul,
>> 
>>  Jan and I talked about this a few times and I started a branch[1] along that idea.
So far all I did was make a copy of the then current Futon tests into
>> test/javascript/test  and started looking at the small handful that fail.
>> 
>>   The browser tests are great (any test is good) but they have too many browser 
dependent quirks, or at least I assume that because of the pleasant surprise
>> one gets when they all run. So I think the goal of these runner tests would be some
sort of official HTTP API suite that's part of "make check". Would you agree?
>> If so I'm happy to take this on.
>> 
>>   Also I've found eunit to be helpful in BigCouch and wondering how hard it would
be to support eunit in couchdb. Having tests in the modules is very good for not
>> only testing but also to help with reading and understanding what the code does.
>> 
>> Bob
>> 
>> 
>> [1] https://github.com/bdionne/couchdb/tree/cli-tests
>> 
>> 
> 
> Bob,
> 
> Exactly what I was thinking. By having our test suite have as little
> code between the socket and the the test as possible we can ensure
> that the tests are testing CouchDB and not some random change in the
> behavior of our favorite web browser. I would definitely expect these
> tests to be part of make check and hence part of the release
> procedure.
> 
> My current half formed thoughts are to basically split our test suite
> into two halves. Tests that are in Erlang and are testing internals,
> and tests that go through the HTTP interface. I love me some Erlang,
> but I've not been think of an elegant way to make it easy to run lots
> of HTTP tests.
> 
> As to eunit, I'm not sure. I'm really not a huge fan of it, especially
> mixing implementation and test code. I know rebar can separate them,
> so its at least possible to get around that. I'd like to have a
> unified environment for Erlang tests though. And TAP at seems like
> it'd be easier to interface with non-Erlang tooling if we ever get
> around to build matrices and what not. But I'm not opposed to it on
> religious grounds if that's what people want to contribute.

OTP ships with eunit_surefire[1] so interfacing with general test infrastructures isn't really
an issue.  I had to dig a little deeper to find a decent TAP consumer for Jenkins and ended
up executing the tests from a Perl script using TAP::Harness::JUnit.  I'll grant that the
TAP output is easier for a human to digest than the eunit output.

I imagine testing the private module functions is a topic for a good flamewar.  I find it
useful, but maybe that's a sign I've got too much logic in a particular module.  Either way,
 a viable alternative to writing unit tests that execute in the browser is something this
project really needs.  Cheers,

Adam

[1]: http://www.erlang.org/doc/man/eunit_surefire.html
Mime
View raw message