couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Doane <jaydo...@apache.org>
Subject Re: [PROPOSAL] Remove `make elixir` from `make check`
Date Sat, 14 Dec 2019 00:50:46 GMT
On Thu, Dec 12, 2019 at 11:27 PM Joan Touzet <wohali@apache.org> wrote:

> Jay, consider this a +1 to remove those if you can get to them before I do.
>

Joan, I don't think I was clear, but when I said
> I'm +1 to removing these tests temporarily.
by "these tests" I actually meant all the elixir tests, not just the two I
mentioned. I don't think we'd get very far by just disabling those two, as
there are many others that fail intermittently as well. I'm +1 to remove
them, and here's a PR in case someone wants it to happen sooner rather than
later:

https://github.com/apache/couchdb/pull/2362

Thanks,
Jay

>
> If this makes the Elixir tests useful again, then it's a good compromise.
>
> -Joan
>
> On 2019-12-13 2:18 a.m., Jay Doane wrote:
> > I've been trying to merge into master for the last 24+ hours, and have
> > restarted the build about 10 times so far, but it has yet to pass.
> >
> > I kept track of failures, and these came up numerous times:
> >
> >   1) test GET /dbname/_design_docs (PartitionDDocTest)
> >       test/elixir/test/partition_ddoc_test.exs:160
> >       ** (RuntimeError) timed out after 10294 ms
> >       code: retry_until(
> >       stacktrace:
> >         (couchdbtest) test/elixir/lib/couch/db_test.ex:301:
> > Couch.DBTest.retry_until/4
> >         test/elixir/test/partition_ddoc_test.exs:163: (test)
> >
> >   1) test All Docs tests (AllDocsTest)
> >       test/elixir/test/all_docs_test.exs:15
> >       ** (RuntimeError) timed out after 30097 ms
> >       code: retry_until(fn ->
> >       stacktrace:
> >         (couchdbtest) test/elixir/lib/couch/db_test.ex:301:
> > Couch.DBTest.retry_until/4
> >         test/elixir/test/all_docs_test.exs:75: (test)
> >
> > which both seem to be timing out.
> >
> > I'm +1 to removing these tests temporarily.
> >
> > Jay
> >
> > On Thu, Dec 12, 2019 at 1:39 PM Joan Touzet <wohali@apache.org> wrote:
> >
> >> Hi again,
> >>
> >> As I've been looking more closely at the CI suite for the Jenkins
> >> transition, I've noticed that our Elixir test cases are actually the
> >> most likely to fail. In 6 consecutive CI runs, 5 runs failed due to
> >> failures in the Elixir suite. (The 6th failed due to a JS test failure.)
> >>
> >> We started the Elixir effort to retire the JS suite. We reached a
> >> decision some months ago to put it into `make check` so that people
> >> would pay attention to its output, and work to fix those tests,
> >> accelerating our chances to get rid of the JS suite.
> >>
> >> Unfortunately, that's not materialised. Our Elixir test porters seem to
> >> have stopped their work for a while now, and no one is systematically
> >> addressing the failures in that suite. I've also heard other developers
> >> mention (via IRC) that some of the test cases hold invalid assumptions
> >> about how CouchDB works, especially with the Erlang-based clustering
> >> code. It sounds to me like the effort needs a full code review.
> >>
> >> With 3.0 around the corner, I want people to be able to trust the output
> >> of `make check` when downloading the tarball. If there is no objection,
> >> when I merge the Erlang version / CI changes on Monday, I will also
> >> comment out the call to `make elixir` as part of make check.
> >>
> >> When the Elixir porting team is more confident in the reliability and
> >> completeness of their work, and we can successfully retire the JS suite,
> >> we can reconsider.
> >>
> >> -Joan "really wanting to see green, but only seeing red" Touzet
> >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message