From dev-return-48939-archive-asf-public=cust-asf.ponee.io@couchdb.apache.org Sat Dec 14 00:51:00 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id B9E3C18064E for ; Sat, 14 Dec 2019 01:50:59 +0100 (CET) Received: (qmail 95257 invoked by uid 500); 14 Dec 2019 00:50:58 -0000 Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@couchdb.apache.org Delivered-To: mailing list dev@couchdb.apache.org Received: (qmail 95245 invoked by uid 99); 14 Dec 2019 00:50:58 -0000 Received: from Unknown (HELO mailrelay1-lw-us.apache.org) (10.10.3.159) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 14 Dec 2019 00:50:58 +0000 Received: from mail-lf1-f50.google.com (mail-lf1-f50.google.com [209.85.167.50]) by mailrelay1-lw-us.apache.org (ASF Mail Server at mailrelay1-lw-us.apache.org) with ESMTPSA id 564704FB3 for ; Sat, 14 Dec 2019 00:50:58 +0000 (UTC) Received: by mail-lf1-f50.google.com with SMTP id y19so542633lfl.9 for ; Fri, 13 Dec 2019 16:50:58 -0800 (PST) X-Gm-Message-State: APjAAAVbEBBJ0ij886Jy9DQhCQ3WCX7WAb5r3NlvIVJCIM6RqMtrlLT5 dMP5Dmd4zzdfrGX0dGzEUd27VCzBKCTSgf1BHh8= X-Google-Smtp-Source: APXvYqxBgt1kiKeqnOzIF0nbTKLI/OMtJ0Rgd+oUaTWSu5VgtTZ1z8F6n0nypRKj/HdMljeWPcBlHkhLPTQa3+fE2z8= X-Received: by 2002:a19:dc1e:: with SMTP id t30mr10720767lfg.34.1576284657409; Fri, 13 Dec 2019 16:50:57 -0800 (PST) MIME-Version: 1.0 References: <23059ff6-8ea3-e154-638a-f4a8e02b0a01@apache.org> <8f9a3449-818f-a09e-dd7d-20425ec65d74@apache.org> In-Reply-To: <8f9a3449-818f-a09e-dd7d-20425ec65d74@apache.org> From: Jay Doane Date: Fri, 13 Dec 2019 16:50:46 -0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PROPOSAL] Remove `make elixir` from `make check` To: dev@couchdb.apache.org Content-Type: multipart/alternative; boundary="00000000000011e63705999f5f03" --00000000000011e63705999f5f03 Content-Type: text/plain; charset="UTF-8" On Thu, Dec 12, 2019 at 11:27 PM Joan Touzet wrote: > Jay, consider this a +1 to remove those if you can get to them before I do. > Joan, I don't think I was clear, but when I said > I'm +1 to removing these tests temporarily. by "these tests" I actually meant all the elixir tests, not just the two I mentioned. I don't think we'd get very far by just disabling those two, as there are many others that fail intermittently as well. I'm +1 to remove them, and here's a PR in case someone wants it to happen sooner rather than later: https://github.com/apache/couchdb/pull/2362 Thanks, Jay > > If this makes the Elixir tests useful again, then it's a good compromise. > > -Joan > > On 2019-12-13 2:18 a.m., Jay Doane wrote: > > I've been trying to merge into master for the last 24+ hours, and have > > restarted the build about 10 times so far, but it has yet to pass. > > > > I kept track of failures, and these came up numerous times: > > > > 1) test GET /dbname/_design_docs (PartitionDDocTest) > > test/elixir/test/partition_ddoc_test.exs:160 > > ** (RuntimeError) timed out after 10294 ms > > code: retry_until( > > stacktrace: > > (couchdbtest) test/elixir/lib/couch/db_test.ex:301: > > Couch.DBTest.retry_until/4 > > test/elixir/test/partition_ddoc_test.exs:163: (test) > > > > 1) test All Docs tests (AllDocsTest) > > test/elixir/test/all_docs_test.exs:15 > > ** (RuntimeError) timed out after 30097 ms > > code: retry_until(fn -> > > stacktrace: > > (couchdbtest) test/elixir/lib/couch/db_test.ex:301: > > Couch.DBTest.retry_until/4 > > test/elixir/test/all_docs_test.exs:75: (test) > > > > which both seem to be timing out. > > > > I'm +1 to removing these tests temporarily. > > > > Jay > > > > On Thu, Dec 12, 2019 at 1:39 PM Joan Touzet wrote: > > > >> Hi again, > >> > >> As I've been looking more closely at the CI suite for the Jenkins > >> transition, I've noticed that our Elixir test cases are actually the > >> most likely to fail. In 6 consecutive CI runs, 5 runs failed due to > >> failures in the Elixir suite. (The 6th failed due to a JS test failure.) > >> > >> We started the Elixir effort to retire the JS suite. We reached a > >> decision some months ago to put it into `make check` so that people > >> would pay attention to its output, and work to fix those tests, > >> accelerating our chances to get rid of the JS suite. > >> > >> Unfortunately, that's not materialised. Our Elixir test porters seem to > >> have stopped their work for a while now, and no one is systematically > >> addressing the failures in that suite. I've also heard other developers > >> mention (via IRC) that some of the test cases hold invalid assumptions > >> about how CouchDB works, especially with the Erlang-based clustering > >> code. It sounds to me like the effort needs a full code review. > >> > >> With 3.0 around the corner, I want people to be able to trust the output > >> of `make check` when downloading the tarball. If there is no objection, > >> when I merge the Erlang version / CI changes on Monday, I will also > >> comment out the call to `make elixir` as part of make check. > >> > >> When the Elixir porting team is more confident in the reliability and > >> completeness of their work, and we can successfully retire the JS suite, > >> we can reconsider. > >> > >> -Joan "really wanting to see green, but only seeing red" Touzet > >> > > > --00000000000011e63705999f5f03--