couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joan Touzet <woh...@apache.org>
Subject Re: 2.0 blocker: new failing test (singular) on Windows
Date Sun, 11 Sep 2016 06:39:51 GMT
Hi Eli and others,

Since my original email I've determined the reason for the failing
test - it's a problem with the test harness itself. What it's trying
to test for is not what's actually being tested. That it only fails
on Windows at present is a vagary of how the BEAM VM is implemented,
and there's no guarantee we might not see the same failures on Linux,
OSX, FreeBSD or any other OS given the way the test is written right
now.

There are two proposals to resolve the problem - an ugly hack of a
sleep() statement or removal of the single failing test from the
suite - and I'm recommending the latter.

The detailed discussion is at 

  https://github.com/apache/couchdb-couch/pull/196

if you want to review the code and understand the problem in depth.

-Joan "why did I stay up until 3AM figuring this out?" Touzet

----- Original Message -----
From: "Eli Stevens (Gmail)" <wickedgrey@gmail.com>
To: "CouchDB Developers" <dev@couchdb.apache.org>, "Joan Touzet" <wohali@apache.org>
Sent: Sunday, September 11, 2016 1:41:37 AM
Subject: Re: 2.0 blocker: new failing tests on Windows

FWVLIW, I'd rather keep the tests active, and have the release happen
with not all tests passing. A failing test doesn't have to be a
release blocker (I mean, this PR is clearly stating as much). Maybe
add a known issue in the release notes or something.

If the policy is "the tests must pass" with an implied "...but it's
okay to comment out any that aren't passing" then it's probably a bad
policy. It's too easy to forgot to re-enable the tests once the
release is out, and hence forget that anything needs to be looked at.

Note: my day job is such that the FDA takes an active interest in the
results of my team's unit tests (the audit a couple of weeks ago was
fun), and so I tend to be a bit more pedantic about these things than
average.

Cheers,
Eli

On Sat, Sep 10, 2016 at 12:36 PM, Joan Touzet <wohali@apache.org> wrote:
> Hi there,
>
> About 3 weeks ago eiri added new tests to the couchdb_os_proc_pool
> suite. Unfortunately, I haven't run the test suite since then, so it is
> only at this late date that I discover two of the tests are failing on
> Windows, blocking my ability to declare the build clean.
>
> You can view sample output of a failed test run here:
>
>   https://issues.apache.org/jira/browse/COUCHDB-3140
>
> I have a PR open to temporarily disable these tests to allow us to
> proceed with a timely 2.0.0 release:
>
>   https://github.com/apache/couchdb-couch/pull/195
>
> The reason for doing so is chiefly one of expediency. At the moment I am
> the only Windows CouchDB developer and I simply don't have the time to
> diagnose what is wrong with either the test or the process manager it is
> testing. I did a bit of playing around with increasing the timeout (and
> configuring eunit to give the tests more time to run than its default
> 5s) but the problem did not go away, suggesting the issue is more subtle
> than one of slow-to-start/stop couchjs processes.
>
> Any help anyone can provide would be greatly appreciated. If I hear
> nothing in the next 24 hours or so, I will merge this PR (and bump
> rebar.config in the main couch repo to pick up the change).
>
> Thanks,
> Joan

Mime
View raw message