From dev-return-48930-archive-asf-public=cust-asf.ponee.io@couchdb.apache.org Thu Dec 12 21:39:04 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 1F2F518061A for ; Thu, 12 Dec 2019 22:39:04 +0100 (CET) Received: (qmail 88743 invoked by uid 500); 12 Dec 2019 21:39:03 -0000 Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@couchdb.apache.org Delivered-To: mailing list dev@couchdb.apache.org Received: (qmail 88731 invoked by uid 99); 12 Dec 2019 21:39:03 -0000 Received: from Unknown (HELO mailrelay1-lw-us.apache.org) (10.10.3.159) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 12 Dec 2019 21:39:03 +0000 Received: from [192.168.1.42] (69-196-165-48.dsl.teksavvy.com [69.196.165.48]) by mailrelay1-lw-us.apache.org (ASF Mail Server at mailrelay1-lw-us.apache.org) with ESMTPSA id 0689410FC for ; Thu, 12 Dec 2019 21:39:03 +0000 (UTC) To: CouchDB Developers From: Joan Touzet Subject: [PROPOSAL] Remove `make elixir` from `make check` Organization: Apache Software Foundation Message-ID: <23059ff6-8ea3-e154-638a-f4a8e02b0a01@apache.org> Date: Thu, 12 Dec 2019 16:39:02 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.9.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Hi again, As I've been looking more closely at the CI suite for the Jenkins transition, I've noticed that our Elixir test cases are actually the most likely to fail. In 6 consecutive CI runs, 5 runs failed due to failures in the Elixir suite. (The 6th failed due to a JS test failure.) We started the Elixir effort to retire the JS suite. We reached a decision some months ago to put it into `make check` so that people would pay attention to its output, and work to fix those tests, accelerating our chances to get rid of the JS suite. Unfortunately, that's not materialised. Our Elixir test porters seem to have stopped their work for a while now, and no one is systematically addressing the failures in that suite. I've also heard other developers mention (via IRC) that some of the test cases hold invalid assumptions about how CouchDB works, especially with the Erlang-based clustering code. It sounds to me like the effort needs a full code review. With 3.0 around the corner, I want people to be able to trust the output of `make check` when downloading the tarball. If there is no objection, when I merge the Erlang version / CI changes on Monday, I will also comment out the call to `make elixir` as part of make check. When the Elixir porting team is more confident in the reliability and completeness of their work, and we can successfully retire the JS suite, we can reconsider. -Joan "really wanting to see green, but only seeing red" Touzet