From dev-return-17455-archive-asf-public=cust-asf.ponee.io@syncope.apache.org Mon Sep 9 09:37:32 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 9C9B4180608 for ; Mon, 9 Sep 2019 11:37:32 +0200 (CEST) Received: (qmail 66435 invoked by uid 500); 9 Sep 2019 09:37:32 -0000 Mailing-List: contact dev-help@syncope.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@syncope.apache.org Delivered-To: mailing list dev@syncope.apache.org Received: (qmail 66424 invoked by uid 99); 9 Sep 2019 09:37:31 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 09 Sep 2019 09:37:31 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 047881A4666 for ; Mon, 9 Sep 2019 09:37:31 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.626 X-Spam-Level: * X-Spam-Status: No, score=1.626 tagged_above=-999 required=6.31 tests=[HK_RANDOM_ENVFROM=0.626, HK_RANDOM_FROM=0.999, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Received: from mx1-he-de.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id k8V5qWdvI9q0 for ; Mon, 9 Sep 2019 09:37:28 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=93.34.9.207; helo=rovere.tirasa.net; envelope-from=misagh.moayyed@tirasa.net; receiver= Received: from rovere.tirasa.net (smtp.tirasa.net [93.34.9.207]) by mx1-he-de.apache.org (ASF Mail Server at mx1-he-de.apache.org) with ESMTPS id C58327D723 for ; Mon, 9 Sep 2019 09:37:27 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by rovere.tirasa.net (Postfix) with ESMTP id 700683C0A0604 for ; Mon, 9 Sep 2019 11:37:21 +0200 (CEST) X-Virus-Scanned: amavisd-new at tirasa.net Received: from rovere.tirasa.net ([127.0.0.1]) by localhost (rovere.tirasa.net [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id duKdf5DGFb7Y for ; Mon, 9 Sep 2019 11:37:21 +0200 (CEST) Received: from rovere.tirasa.net (rovere.tirasa.net [172.25.0.101]) by rovere.tirasa.net (Postfix) with ESMTP id 52DAA3C055B94 for ; Mon, 9 Sep 2019 11:37:21 +0200 (CEST) Date: Mon, 9 Sep 2019 11:37:21 +0200 (CEST) From: Misagh Moayyed To: dev Message-ID: <88911877.4095.1568021841214.JavaMail.zimbra@tirasa.net> Subject: Reorganizing Travis CI with build stages MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [192.168.0.254] X-Mailer: Zimbra 8.8.12_GA_3844 (ZimbraWebClient - GC76 (Mac)/8.8.12_GA_3844) Thread-Index: kLTlghyAwcmnwmVdcvJWvoBRLJlJ/w== Thread-Topic: Reorganizing Travis CI with build stages Hello all, In discussing git workflows with Francesco, I have learned that the Travis CI process is not actually running a comprehensive set of tests and checks (such as checkstyle, etc) that would be valuable for pull requests and contributors, possibly because such jobs take a long time, more than what is allowed by Travis CI. I'd like to propose to reorganize the CI build process to take advantage of the build stages feature [1]. This [2] demonstrates an example where a given build is broken down into multiple stages, each of which has numerous jobs. A key thing to note in this configuration, as I understand, is that a given CI build can have an *unlimited* number of stages, each with an unlimited number of jobs. While each job cannot exceed a certain timeout value [3], the collective overall build stage can go on forever in theory. Also, stages do depend on each other such that failures in one would halt the build. A typical example would be: "Run the build, and if everything compiles, run tests, then publish snapshots, etc". I think this is quite valuable for contributors who, specially, work on multiple JIRAs at the same time. A sample workflow is to add a change, do some sanity tests and then push to let CI run the full build suite. While it's busy kicking off the build, one could switch to the next JIRA to repeat the same process, and as such hop back and forth between branches and JIRAs to parallelize work activity. Ultimately, the CI is the canonical reference for all changes and this would avoid the problem of "but it works on my machine" and of course, saves quite a bit of time. (A full build for me takes about 20-30 minutes, if the stars align). How do folks feel about this proposal? [1] https://docs.travis-ci.com/user/build-stages/ [2] https://travis-ci.org/apereo/cas/builds/581042662 [3] The good folks at Travis CI in the past have been kind enough to increase the timeout from 50 minutes to 180 minutes and they might be willing to make the same change for syncope. Larger tasks need to be broken down to separate jobs, of course. --Misagh