couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Lehnardt <...@apache.org>
Subject Re: [PROPOSAL] Remove `make elixir` from `make check`
Date Mon, 16 Dec 2019 09:39:30 GMT


> On 14. Dec 2019, at 18:29, Garren Smith <garren@apache.org> wrote:
> 
> I’m a bit nervous to remove the full elixir test suite. The only partition
> tests are in the elixir suite. So we would have no test coverage for any of
> that if we remove this. Given that we want to make this the best CouchDB
> release ever it feels counter-intuitive to them be removing tests.
> 
> I also think there is a few JS tests we have disabled because they are
> covered in Elixir so we would need to re-enable those in the same PR that we remove the
elixir tests.

That hasn’t landed yet afaict[1]: but I suggest we get this going to create some progress
on the deprecate-js-tests train.

Best
Jan
—
[1]: https://github.com/apache/couchdb/commit/20e7b2e0cf496c925e99c80befa8b50fd69b5017



> 
> Cheers
> Garren
> 
> On Sat, Dec 14, 2019 at 2:51 AM Jay Doane <jaydoane@apache.org> wrote:
> 
>> On Thu, Dec 12, 2019 at 11:27 PM Joan Touzet <wohali@apache.org> wrote:
>> 
>>> Jay, consider this a +1 to remove those if you can get to them before I
>> do.
>>> 
>> 
>> Joan, I don't think I was clear, but when I said
>>> I'm +1 to removing these tests temporarily.
>> by "these tests" I actually meant all the elixir tests, not just the two I
>> mentioned. I don't think we'd get very far by just disabling those two, as
>> there are many others that fail intermittently as well. I'm +1 to remove
>> them, and here's a PR in case someone wants it to happen sooner rather than
>> later:
>> 
>> https://github.com/apache/couchdb/pull/2362
>> 
>> Thanks,
>> Jay
>> 
>>> 
>>> If this makes the Elixir tests useful again, then it's a good compromise.
>>> 
>>> -Joan
>>> 
>>> On 2019-12-13 2:18 a.m., Jay Doane wrote:
>>>> I've been trying to merge into master for the last 24+ hours, and have
>>>> restarted the build about 10 times so far, but it has yet to pass.
>>>> 
>>>> I kept track of failures, and these came up numerous times:
>>>> 
>>>>  1) test GET /dbname/_design_docs (PartitionDDocTest)
>>>>      test/elixir/test/partition_ddoc_test.exs:160
>>>>      ** (RuntimeError) timed out after 10294 ms
>>>>      code: retry_until(
>>>>      stacktrace:
>>>>        (couchdbtest) test/elixir/lib/couch/db_test.ex:301:
>>>> Couch.DBTest.retry_until/4
>>>>        test/elixir/test/partition_ddoc_test.exs:163: (test)
>>>> 
>>>>  1) test All Docs tests (AllDocsTest)
>>>>      test/elixir/test/all_docs_test.exs:15
>>>>      ** (RuntimeError) timed out after 30097 ms
>>>>      code: retry_until(fn ->
>>>>      stacktrace:
>>>>        (couchdbtest) test/elixir/lib/couch/db_test.ex:301:
>>>> Couch.DBTest.retry_until/4
>>>>        test/elixir/test/all_docs_test.exs:75: (test)
>>>> 
>>>> which both seem to be timing out.
>>>> 
>>>> I'm +1 to removing these tests temporarily.
>>>> 
>>>> Jay
>>>> 
>>>> On Thu, Dec 12, 2019 at 1:39 PM Joan Touzet <wohali@apache.org> wrote:
>>>> 
>>>>> Hi again,
>>>>> 
>>>>> As I've been looking more closely at the CI suite for the Jenkins
>>>>> transition, I've noticed that our Elixir test cases are actually the
>>>>> most likely to fail. In 6 consecutive CI runs, 5 runs failed due to
>>>>> failures in the Elixir suite. (The 6th failed due to a JS test
>> failure.)
>>>>> 
>>>>> We started the Elixir effort to retire the JS suite. We reached a
>>>>> decision some months ago to put it into `make check` so that people
>>>>> would pay attention to its output, and work to fix those tests,
>>>>> accelerating our chances to get rid of the JS suite.
>>>>> 
>>>>> Unfortunately, that's not materialised. Our Elixir test porters seem
>> to
>>>>> have stopped their work for a while now, and no one is systematically
>>>>> addressing the failures in that suite. I've also heard other
>> developers
>>>>> mention (via IRC) that some of the test cases hold invalid assumptions
>>>>> about how CouchDB works, especially with the Erlang-based clustering
>>>>> code. It sounds to me like the effort needs a full code review.
>>>>> 
>>>>> With 3.0 around the corner, I want people to be able to trust the
>> output
>>>>> of `make check` when downloading the tarball. If there is no
>> objection,
>>>>> when I merge the Erlang version / CI changes on Monday, I will also
>>>>> comment out the call to `make elixir` as part of make check.
>>>>> 
>>>>> When the Elixir porting team is more confident in the reliability and
>>>>> completeness of their work, and we can successfully retire the JS
>> suite,
>>>>> we can reconsider.
>>>>> 
>>>>> -Joan "really wanting to see green, but only seeing red" Touzet
>>>>> 
>>>> 
>>> 
>> 

-- 
Professional Support for Apache CouchDB:
https://neighbourhood.ie/couchdb-support/


Mime
View raw message