Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
Received-SPF: pass (athena.apache.org: domain of paul.joseph.davis@gmail.com
 designates 209.85.132.249 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type:content-transfer-encoding;
        b=tRqcjpe7iRjIGd6uKMsCjk5gFLZbT1UcDUpBKjDNzQq5eaB576D54evD0e8zLNf00M
         6oYhwSwdsOq5Cul4ltPOOSFE2I5LBw+aScrgMVbjqLXy9KvOXzxABPQkiAYHOEtOSB9x
         4IqRNxZTQTLp2eaQKk3GcG6/3EFiaFD5Qf7Uw=
MIME-Version: 1.0
In-Reply-To: <e282921e0902131052j1b47b6c7v77cec9c177c35df@mail.gmail.com>
References: <D574CA7F-2441-4342-9382-1AD8A20395EB@apache.org>
	 <e2111bbb0902131030q776b5a4fje5f8230286aa4884@mail.gmail.com>
	 <e282921e0902131052j1b47b6c7v77cec9c177c35df@mail.gmail.com>
Date: Fri, 13 Feb 2009 16:02:00 -0500
Message-ID: <e2111bbb0902131302t2c4232bci79264e49ae43df56@mail.gmail.com>
Subject: Re: Ideas for Changes to the Test Suite
From: Paul Davis <paul.joseph.davis@gmail.com>
To: dev@couchdb.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

On Fri, Feb 13, 2009 at 1:52 PM, Chris Anderson <jchris@apache.org> wrote:
> On Fri, Feb 13, 2009 at 10:30 AM, Paul Davis
> <paul.joseph.davis@gmail.com> wrote:
>>
>> 2. I'd still argue that we shouldn't be using a browser as our native
>> test runner. We'd have to give up the little green check marks that
>> make us all feel warm and fuzzy when tests pass, but the browser is a
>> huge ass confounding variable.
>
> I know for sure that in-browser tests are a big part of what brought
> me to CouchDB. They tell the story of a web-native database in a way
> that nothing else can really touch. They also make it *incredibly
> easy* for newcomers to contribute.
>

Heh. I actually took a while to come around to the idea of the pure
couch application. I see your point and I raise you though. I'm all
for some snazzy looking tests in the browser, but I would argue that
if we went with doing the basics for browser interaction we'd be set.
Ie, CRUD operations, checking out _uuids and _bulk_docs etc etc. I'm
thinking the Futon test suite could be similar to what it is, but
instead of  testing your couchdb install, it's testing your browser's
conformance to working with CouchDB. Hell, we could even start a wiki
page for "These browsers have passed the CouchDB test suite" type of
thing. or run one of those render things on it too.

>>To me, a proper test suite would be run
>> from directly from the command line. We have the hacked together test
>> runner, but not many people seem to use it regularly because we have
>> the fancy green check marks.
>>
>
> I think we're starting to feel the lack of Erlang unit tests. They
> sure would have helped me in my last few patches, and they'd make a
> decent beginning for documenting our native Erlang API.
>
> I think once we get the few Erlang tests that are already written,
> integrated into the build, and make it easy to add new ones, they will
> become the primary command-line test suite.
>

eunit tests are definitely important as well, but they're testing
something completely different. I tend to look at my testing as a
large tree. unit tests are at the very bottom making sure specific
functions are working, then we test different combinations, and etc
etc until we're at the top testing the  user API. Purists hate me for
saying that because they think that *everything* should be tested
independently. I hate purists though so it all evens out.

> Chris
>
> --
> Chris Anderson
> http://jchris.mfdz.com
>

Also, s/trunk/host/ in my last.

HTH,
Paul Davis