cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sergey Levitskiy <>
Subject Re: [GitHub] cloudstack issue #1658: Added an additional JSON diff output to the ApiXmlDo...
Date Mon, 12 Sep 2016 20:09:46 GMT
@jburwell This looks like an awesome system which will improve ACS stability big time. Let
me know if we can assist in any way. 

On 9/12/16, 12:42 PM, "John Burwell" <> wrote:

    We are working on a full automated build pipeline that will test against a wide range
of configurations.  Ideally, the pipeline will work as follows:
         1. PR is submitted -> the PR is built, unit tested, and run through static analysis.
 Smoke tests are run on a matrix of configurations
         2. The PR is reviewed by two or more committers
         3. When a PR commit receives two or more LGTMs and no -1s, it is enqueued to be merged
to the target branch
         4. When it’s PR’s turn to be merged, it merged against the target release branch
and rebuilt (unit tests + static analysis), packaged, and tested (smoke test + a set of component
tests) on a matrix of configurations
         5. If the PR needs to be forward merged, it is enqueued for the next branch and Step
4 is re-executed
    In this system, only one PR would be actively merged at a time.  Steps 4 and 5 are repeated
until the PR is merged to master.  This process not only ensures that the PR is adequately
reviewed and tested, and it also detects introduced by merges of adjacent PRs.  blueorganatan
is first step in the process which provides a way for indicate that a PR is ready for the
next step.
    We are working towards such a system, but, obviously, there are a lot of pieces we need
to refine/develop and integrate.  I am working up a release management roadmap to capture
these ideas in more detail which I hope publish in the near future,
    53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
    On Sep 12, 2016, at 3:29 PM, Sergey Levitskiy <> wrote:
    > @swill @jburwell From my, non-committer, point of view introducing fully automated
test system integrated with PR submission and leveraging complete list of supported hypervisors
is the key to stability. I really like blueorangutain idea and if this can be brought back
and also cover full set of integration tests along with packaging that would be awesome. When
we develop PR unfortunately we can only test it only with a certain set of hypervisors and
network configs so although giving test LGTM it doesn’t cover all possible integration points.
I believe most other test results people execute manually and post on PRs are the same.
    > On 9/12/16, 11:38 AM, "jburwell" <> wrote:
    > Even 
    >    Github user jburwell commented on the issue:
    >        @swill where hardware are being varies by PR.  In some cases, we have people
running them in their labs and reporting results.  Other cases, it's blueorangatan going through
ShapeBlue Jenkins + trillian.  But yes, PRs are getting tested on hardware.
    >        (Sadly, we are still recovering from a dead storage array, so blueorangatan
has been out of commission for a little bit),
    >    ---
    >    If your project is set up for it, you can reply to this email and have your
    >    reply appear on GitHub as well. If your project does not have this feature
    >    enabled and wishes so, or if the feature is enabled but not working, please
    >    contact infrastructure at or file a JIRA ticket
    >    with INFRA.
    >    ---

View raw message