From dev-return-4849-archive-asf-public=cust-asf.ponee.io@mxnet.incubator.apache.org Tue Nov 20 17:05:41 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id DDEAB180675 for ; Tue, 20 Nov 2018 17:05:40 +0100 (CET) Received: (qmail 94945 invoked by uid 500); 20 Nov 2018 16:05:40 -0000 Mailing-List: contact dev-help@mxnet.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@mxnet.incubator.apache.org Delivered-To: mailing list dev@mxnet.incubator.apache.org Received: (qmail 94456 invoked by uid 99); 20 Nov 2018 16:05:39 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 20 Nov 2018 16:05:39 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id EE077D0C22 for ; Tue, 20 Nov 2018 16:05:38 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.339 X-Spam-Level: X-Spam-Status: No, score=0.339 tagged_above=-999 required=6.31 tests=[DKIMWL_WL_MED=-1.46, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=googlemail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id kFeYI-l8UVbb for ; Tue, 20 Nov 2018 16:05:37 +0000 (UTC) Received: from mail-qt1-f169.google.com (mail-qt1-f169.google.com [209.85.160.169]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 5B9DB5F563 for ; Tue, 20 Nov 2018 16:05:37 +0000 (UTC) Received: by mail-qt1-f169.google.com with SMTP id d19so536689qtq.9 for ; Tue, 20 Nov 2018 08:05:37 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=LhHVFLfmBoXqQlQI1A7CBXzv78xw5CdEMQfRgAUPiYE=; b=n07byubHiD0GZEtvK2CMJ0bakpz1oUPQ4dKjYH3/1aphiUfhnDN2KxenAlBuvbQr+d GY+Lv2ZWhQooA13UtGpZTA2GSZOvpfJcRVnbDXYXC0Fwa79tsiPgw76ZekwL5t20cpem J8lx/A4yJOchTiSWf0Ez14A3Hy5J6NAkdhbUr9v4AZr2SPZQ3nGvi48Oh5a61EnauyEc i5oawOzC0P5Cc8JwhFm29REXRLgCKIir+OG39fdolYT8eSJvAg3YS9qPhtG9vp4uk2Ah +WWqXqQReN5XgpMi0ruMW+63JRzYzmi/TYFrOfb7QnFmI0ofH0uCG0i+Xv3zCMsMHi1X 66NA== X-Gm-Message-State: AGRZ1gLlYOsI1AcAflBMKlDVuQlQ5M/b1zvKNgQu3wjqEJPNxU+TnJ8B lHHemx4gjq//Jx6kelEODhoZlvn+jNA2/g8Eu+6d69T8 X-Google-Smtp-Source: AFSGD/VMevkA6+1YjfXF5ttfOCOTfDKqIluBYxb1y/2jLXsvU2hcVwzVoFE8XlR7nHoqLj/wsR1Jqpa+oEbwhU4c2io= X-Received: by 2002:aed:3b76:: with SMTP id q51mr2384906qte.229.1542729930700; Tue, 20 Nov 2018 08:05:30 -0800 (PST) MIME-Version: 1.0 From: Marco de Abreu Date: Tue, 20 Nov 2018 17:04:54 +0100 Message-ID: Subject: Splitting Jenkins pipelines - stop changes to Jenkinsfiles! To: dev@mxnet.incubator.apache.org Content-Type: multipart/alternative; boundary="00000000000080ff55057b1acd11" --00000000000080ff55057b1acd11 Content-Type: text/plain; charset="UTF-8" Hello, we ran into issues around the maximum filesize of the Jenkinsfile a few times already. In order to resolve this issue, I'd like to combine this with some refactors I have planned for quite some time. The idea is basically to move away from one big Jenkinsfile and instead split it into separate jobs that run in parallel and report their status individually. Besides avoiding the size restriction, this will greatly speed up the PR validation process by reducing the critical path. Instead of having to wait for every single step within a stage to finish before the next stage (e.g. tests) is getting executed, these pipelines would now be able to move forward individually. I'm still in the process of refactoring and can't provide any numbers or documentation at this time, but I would like to announce this early on to avoid conflicts: Since I will remove the original Jenkinsfile, this might cause conflicts with ongoing efforts that try to change the Jenkinsfile. This poses the risk that I might forget to port a change. Thus, I'd like to ask all contributors to wait with changes of Jenkinsfile and would like to request fellow-committers to wait with merging any Jenkinsfile-related PRs until further notice. I expect to finish this refactor until the end of the week. Please don't hesitate to ask if you've got further questions. Please excuse any caused inconveniences. Best regards, Marco --00000000000080ff55057b1acd11--