geode-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kenneth Howe <kh...@pivotal.io>
Subject Re: [DISCUSS] CI improvements
Date Fri, 06 Oct 2017 17:05:25 GMT
+1 - Reduce the noise level for analyzing CI results

> On Oct 6, 2017, at 9:49 AM, Udo Kohlmeyer <ukohlmeyer@pivotal.io> wrote:
> 
> +1 Switch... parallel runs would be safest
> 
> On Fri, Oct 6, 2017 at 9:42 AM, Jianxia Chen <jchen21@apache.org> wrote:
> 
>> +1 to switch to Concourse
>> +1 As a first step we could run both CI systems side-by-side and see how
>> the Concourse approach works for our project.
>> 
>> On Fri, Oct 6, 2017 at 7:08 AM, Anthony Baker <abaker@pivotal.io> wrote:
>> 
>>> Hi all,
>>> 
>>> I’d like to propose the following that we switch our continuous
>>> integration (CI) system from Jenkins [1] to Concourse [2].  I suggest
>>> this because we continue to experience a significant number of
>>> environmental-related test failures.
>>> 
>>> These issues include CPU interference from other Jenkins jobs on the
>>> same host, running out of disk space, port conflicts, and other
>>> gremlins.  The net effect is that we are only getting 1-2 successful
>>> builds per month.  Certainly not all test failures can be traced back
>>> to environmental issues.  However, internal testing on isolated VM’s
>>> shows a combined success rate of about 3X higher compared to ASF
>>> Jenkins for the same tests.  This is still definitely NotAwesome, but
>>> removing environmental factors will let us focus on stabilizing flaky
>>> tests.
>>> 
>>> Concourse is an Apache-licensed open source CI system based on
>>> pipelines.  The pipelines are defined in a YML file containing job
>>> definitions—inputs, outputs, resources, and tasks.  A task is simply a
>>> bash script that returns 0/1 for success/failure.  A web UI displays
>>> build status.  Importantly, each job runs inside an isolated
>>> container.  The containers are load-balanced across a pool of workers.
>>> For an example of a build pipeline, see [3] for the pipeline used to
>>> build concourse itself.
>>> 
>>> A Concourse environment is deployed and managed in cloud environments
>>> through bosh [4].  Pivotal has agreed to donate AWS and/or GCP compute
>>> and storage resources as well as manage the infrastructure.  These
>>> project resources would be available for use by all committers and
>>> community members regardless of corporate affiliations.  Note that
>>> AFAIK there is no explicit requirement to host CI on ASF
>>> infrastructure—unlike for critical project resources such as source
>>> code, mailing lists, and issue tracking.
>>> 
>>> The source for the pipeline and job scripts would reside within the
>>> geode-* repos.  Geode committers would be able to modify those, same
>>> as with our .travis.yml scripts.  All test results and build artifacts
>>> would be publicly viewable just like with our Jenkins build output
>>> today.  Requests for admin assistance would go through the dev@geode
>>> mailing list.
>>> 
>>> Thoughts?  As a first step we could run both CI systems side-by-side
>>> and see how the Concourse approach works for our project.
>>> 
>>> Thanks,
>>> Anthony
>>> 
>>> 
>>> [1] https://builds.apache.org/job/Geode-nightly/
>>> [2] https://concourse.ci
>>> [3] https://ci.concourse.ci
>>> [4] https://bosh.io
>>> 
>> 
> 
> 
> 
> -- 
> Kindest Regards
> -----------------------------
> *Udo Kohlmeyer* | *Snr Solutions Architect* |*Pivotal*
> *Mobile:* +61 409-279-160 | ukohlmeyer@pivotal.io
> <http://www.gopivotal.com/>
> www.pivotal.io


Mime
View raw message