couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Lehnardt <>
Subject Re: Futon Test Suite
Date Thu, 11 Aug 2011 11:40:22 GMT

On Aug 11, 2011, at 1:23 PM, Robert Newson wrote:

> Somewhat OT but you reminded me of something.
> The 'verify my install' on trunk strikes me as a bad idea. If someone
> reports a bug and says that the verify passes, I'm always going to ask
> them to run the test suite too.

That is the intended way.

> It exists, it seems, because the test suite is too slow.

It exists because the test suite is too brittle and does way too much to test whether an installation
is generally sound.

> As Paul has noted, some tests are slow not because
> they are doing a lot of tests, as some have claimed, but because they
> unilaterally wait for several seconds per iteration. That is, they
> just aren't that well written.
> Can we make the test suite faster, more reliable, and less intrusive
> (i.e, not blow away your admins, etc)?

Yes, but not without significant effort that in the past three years where we all agreed that
the extensive test suite is a bad idea as is nobody has shouldered. So my hopes are low.


> B.
> On 11 August 2011 11:57, Jason Smith <> wrote:
>> On Thu, Aug 11, 2011 at 4:29 PM, Paul Davis <> wrote:
>>> All very good except this one paragraph. The CouchDB definitely should
>>> not be expected to run with an intermediary server. If an intermediary
>>> is broken, its quite all right that we engineer paths around
>>> brokenness, but that's secondary by many orders of magnitude to
>>> asserting the behavior of CouchDB's API.
>> I buy that.
>> What about if and when the test suite splits into "confirm the
>> install" vs. a comprehensive unit tester? I suppose the comprehensive
>> test can demand a direct (or even null?) connection. But can "confirm
>> the install" be so bossy?
>> I guess the answer is also "yes." You are confirming end-to-end
>> (browser-to-server) functionality. If the proxy is breaking
>> expectations, then indeed you *want* to see a big red warning.
>> The only problem is this:
>> CouchDB developers are sitting pretty in the United States, maybe
>> Western Europe: basically the center of the universe. Everything is
>> fast, packets never drop, latency is an afterthought. Meanwhile,
>> across Latin America, Russia, China, and South and Southeast Asia
>> (that I know of, from support tickets), EDGE networking is everywhere.
>> Packets always drop. Non-standard transparent proxies are everywhere.
>> They are built in to ISPs. You cannot avoid them. Unlike the West,
>> HTTPS is not magic. There is huge latency, the handshakes take many
>> seconds to complete.
>> On stardate 47805.1, Commander Benjamin Sisko famously said:
>>> On Earth, there is no poverty, no crime, no war. You look out the
>>> window of Starfleet Headquarters and you see paradise. Well, it's
>>> easy to be a saint in paradise, but the Maquis do not live in paradise.
>>> Out there in the Demilitarized Zone, all the problems haven't been
>>> solved yet. Out there, there are no saints — just people. Angry, scared,
>>> determined people who are going to do whatever it takes to survive,
>>> whether it meets with Federation approval or not!
>> It is easy to be a saint in paradise. This applies to CouchDB and
>> Couch apps. On many ISPs, the non-standard, "transparent" proxies are
>> inescapable. But yet, web applications always work. The LA Times
>> works, works, random wordpress blogs work, everything I've
>> ever clicked from Hacker News works. But yet Couch apps are joke. It
>> seems nothing is ever cached when it should be, and everything is
>> always cached when it shouldn't be. Authentication, in particular,
>> fails because caching proxies don't care about DELETE /_session.
>> I have no immediately actionable advice here, but my goal is just to
>> point out that, at some point, demanding standards-compliance becomes
>> bigotry, and if we are too ideologically pure, we risk alienating a
>> larger development community. And read those list of countries again.
>> These communities stand to gain the most from CouchDB and the p2p web.
>> --
>> Iris Couch

View raw message