cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Semb Wever (Jira)" <j...@apache.org>
Subject [jira] [Created] (CASSANDRA-16006) Parallelise Jenkins dtests
Date Fri, 31 Jul 2020 15:28:00 GMT
Michael Semb Wever created CASSANDRA-16006:
----------------------------------------------

             Summary: Parallelise Jenkins dtests
                 Key: CASSANDRA-16006
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-16006
             Project: Cassandra
          Issue Type: Improvement
          Components: CI
            Reporter: Michael Semb Wever


Currently dtests in Jenkins take ~10 hours.

Using the Jenkins Matrix plugin these jobs can be split into smaller lists of dtests and run
in parallel. This is the approach CircleCI takes.

This approach was [trialed|https://github.com/apache/cassandra-builds/pull/29] with the dtest-upgrade
jobs (which are not yet part of the branch pipelines, and haven't previously worked at all
due to their duration).

In addition to the Matrix plugin, the Priority-Sorter and Matrix Reloaded plugins also needed
to be added.

The splits will occupy all executors, and multiple builds will lead to a long build queue.
More important builds (artifacts and unit tests) need a way to be prioritised in such saturated
situations.

Splits can fail for silly reasons (false-positive), like full /tmp disks, or connectivity
issues between the donated agent servers. The Matrix Reloaded plugin makes it easy to rebuilt
just those failed splits.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org


Mime
View raw message