Mailing-List: contact dev-help@cloudstack.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cloudstack.apache.org
Received-SPF: pass (athena.apache.org: domain of darren.s.shepherd@gmail.com
 designates 209.85.212.181 as permitted sender)
MIME-Version: 1.0
Date: Sun, 27 Oct 2013 08:51:26 -0700
Message-ID: 
 <CAMCEDC2Q8-y5gKYMeOCmWfjM=x2SfoaXk41aBw-k4D4rbh5HTw@mail.gmail.com>
Subject: Tiered Quality
From: Darren Shepherd <darren.s.shepherd@gmail.com>
To: "dev@cloudstack.apache.org" <dev@cloudstack.apache.org>
Content-Type: text/plain; charset=ISO-8859-1

I don't know if a similar thing has been talked about before but I
thought I'd just throws this out there.  The ultimate way to ensure
quality is that we have unit test and integration test coverage on all
functionality.  That way somebody authors some code, commits to, for
example, 4.2, but then when we release 4.3, 4.4, etc they aren't on
the hook to manually tests the functionality with each release.  The
obvious nature of a community project is that people come and go.  If
a contributor wants to ensure the long term viability of the
component, they should ensure that there are unit+integration tests.

Now, for whatever reason whether good or bad, it's not always possible
to have full integration tests.  I don't want to throw down the gamut
and say everything must have coverage because that will mean some
useful code/feature will not get in because of some coverage wasn't
possible at the time.

What I propose is that for every feature or function we put it in a
tier of what is the quality of it (very similar to how OpenStack
qualifies their hypervisor integration).  Tier A means unit test and
integration test coverage gates the release.  Tier B means unit test
coverage gates the release.  Tier C mean who knows, it compiled.  We
can go through and classify the components and then as a community we
can try to get as much into Tier A as possible.

Darren