Mailing-List: contact cloudstack-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: cloudstack-dev@incubator.apache.org
Received-SPF: pass (athena.apache.org: domain of Alex.Huang@citrix.com
 designates 66.165.176.89 as permitted sender)
From: Alex Huang <Alex.Huang@citrix.com>
To: "cloudstack-dev@incubator.apache.org"
	<cloudstack-dev@incubator.apache.org>
Date: Tue, 5 Mar 2013 10:42:11 -0800
Subject: [PROPOSAL] BVT for CloudStack checkins
Thread-Topic: [PROPOSAL] BVT for CloudStack checkins
Thread-Index: Ac4ZzW8YmrfH9JmjRwavufb4jF18iw==
Message-ID: 
 <B1DF26ECC0458748AC97CECE2DA98D41013175649781@SJCPMAILBOX01.citrite.net>
Accept-Language: en-US
Content-Language: en-US
acceptlanguage: en-US
Content-Type: multipart/alternative;
	boundary="_000_B1DF26ECC0458748AC97CECE2DA98D41013175649781SJCPMAILBOX_"
MIME-Version: 1.0

--_000_B1DF26ECC0458748AC97CECE2DA98D41013175649781SJCPMAILBOX_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Hi All,

As most of you are aware, the master branch keeps getting broken by checkin=
s for various reasons.  Committers need to be more responsible about their =
checkins but I don't think we can depend on that happening.  There are vari=
ous reasons.  The most obvious to me is that granting committership is not =
based on code competency.  (And I don't think it should.)  Given that we ne=
ed to build a BVT system for ensure that checkins do not break the branch.

Here's my proposal:

Existing components that we'll use.

-          Citrix has contributed its testing to Apache.

-          Apache CloudStack has already a simulator that's been used for s=
cale testing.

-          Marvin

-          DevCloud-kvm

Work Proposal:

-          Convert the Citrix testing into three phases:

o   Setup

o   Test

o   Verify

-          Add a Setup and Verify phase for the simulator

-          Add all of the agent commands necessary for the simulator to pas=
s the testing.

-          Add a Setup and Verify phase for devCloud-kvm

-          Add two more profiles to pom

o   Checkin-test-with-simulator: Runs the testing against the simulator

o   Checkin-test-with-devCloud: Runs the testing against devcloud

-          All of the profiles will attempt to also check the merge list th=
at Chip has proposed.'

-          We will also change marvin to easily add zones with actual hardw=
are.  It will be based on a data driven document to do the setup.

For a developer to checkin:

-          S/he must writes marvin tests for their feature and add it to th=
e BVT.

-          S/he must run the checkin tests to verify everything works.

-          If the feature contains a hardware/vpx component, simulation mus=
t be added.

At this point, everything is about the developer writing in their feature b=
ranch and merging in.

On infrastructure side:

-          We'll setup continuous BVT based on the simulator.

-          I again push that we must use Gerrit to test the code before it =
gets merge into the branch but I'll leave that for someone else to do that.

Let me know what you guys think.  I'll probably break out a bvt branch to w=
ork on this.  Anyone want to join me?

--Alex


--_000_B1DF26ECC0458748AC97CECE2DA98D41013175649781SJCPMAILBOX_--