Mailing-List: contact dev-help@cloudstack.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cloudstack.apache.org
Received-SPF: pass (nike.apache.org: domain of santhosh.edukulla@citrix.com
 designates 103.14.252.240 as permitted sender)
From: Santhosh Edukulla <santhosh.edukulla@citrix.com>
To: "dev@cloudstack.apache.org" <dev@cloudstack.apache.org>
Subject: RE: Tiered Quality
Thread-Topic: Tiered Quality
Thread-Index: 
 AQHO0yyEf4+0dEUe6kSbiwVCJWhlspoIT20AgAAQKwCAADPsAIAAc0gAgADXQoCAAJwuTA==
Date: Mon, 28 Oct 2013 19:08:25 +0000
Message-ID: 
 <CB688D716959224C95995F52DE5E82420EBA1AA9@SINPEX01CL02.citrite.net>
References: 
 <CAMCEDC2Q8-y5gKYMeOCmWfjM=x2SfoaXk41aBw-k4D4rbh5HTw@mail.gmail.com>
	<CAKRHFXViC0LDd-eXy-5dkwwmEGCaTfwcdSEuVLTH3FSDQ8cHdw@mail.gmail.com>
	<E2F4197B-6C81-4E1C-B20C-37D4B869E2EA@gmail.com>
	<CAKRHFXVgvGOoW7cUQPq6W+EVXUDZt-t_bSH1seUQjQ+=Qg6w_g@mail.gmail.com>
	<20131028045322.GA1593@cloud-2.local>,<CAKRHFXX+-0a=NLpAqZ=QtLi=pHc=E7mnXastTz20b8vtmiCOOg@mail.gmail.com>
In-Reply-To: 
 <CAKRHFXX+-0a=NLpAqZ=QtLi=pHc=E7mnXastTz20b8vtmiCOOg@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

1.It seems we already have a code coverage numbers using sonar as below. It=
 currently shows only the numbers for unit tests.

https://analysis.apache.org/dashboard/index/100206

2. The below link has an explanation for using it for both integration and =
unit tests.

http://docs.codehaus.org/display/SONAR/Code+Coverage+by+Integration+Tests+f=
or+Java+Project

3. Many links suggests it has good decision coverage facility compared to o=
ther coverage tools.

http://onlysoftware.wordpress.com/2012/12/19/code-coverage-tools-jacoco-cob=
ertura-emma-comparison-in-sonar/

Regards,
Santhosh
________________________________________
From: Laszlo Hornyak [laszlo.hornyak@gmail.com]
Sent: Monday, October 28, 2013 1:43 PM
To: dev@cloudstack.apache.org
Subject: Re: Tiered Quality

Sonar already tracks the unit test coverage. It is also able to track the
integration test coverage, however this might be a bit more sophisticated
in CS since not all hardware/software requirements are available in the
jenkins environment. However, this could be a problem in any environment.


On Mon, Oct 28, 2013 at 5:53 AM, Prasanna Santhanam <tsp@apache.org> wrote:

> We need a way to check coverage of (unit+integration) tests. How many
> lines of code hit on a deployed system that corresponds to the
> component donated/committed. We don't have that for existing tests so
> it makes it hard to judge if a feature that comes with tests covers
> enough of itself.
>
> On Sun, Oct 27, 2013 at 11:00:46PM +0100, Laszlo Hornyak wrote:
> > Ok, makes sense, but that sounds like even more work :) Can you share t=
he
> > plan on how will this work?
> >
> >
> > On Sun, Oct 27, 2013 at 7:54 PM, Darren Shepherd <
> > darren.s.shepherd@gmail.com> wrote:
> >
> > > I think it can't be at a component level because components are too
> large.
> > >  It needs to be at a feature for implementation level.  For example,
> live
> > > storage migration for xen and live storage migration for kvm (don't
> know if
> > > that's a real thing) would be two separate items.
> > >
> > > Darren
> > >
> > > > On Oct 27, 2013, at 10:57 AM, Laszlo Hornyak <
> laszlo.hornyak@gmail.com>
> > > wrote:
> > > >
> > > > I believe this will be very useful for users.
> > > > As far as I understand someone will have to qualify components. Wha=
t
> will
> > > > be the method for qualification? I do not think simply the test
> coverage
> > > > would be right. But then if you want to go deeper, then you need a
> bigger
> > > > effort testing the components.
> > > >
> > > >
> > > >
> > > > On Sun, Oct 27, 2013 at 4:51 PM, Darren Shepherd <
> > > > darren.s.shepherd@gmail.com> wrote:
> > > >
> > > >> I don't know if a similar thing has been talked about before but I
> > > >> thought I'd just throws this out there.  The ultimate way to ensur=
e
> > > >> quality is that we have unit test and integration test coverage on
> all
> > > >> functionality.  That way somebody authors some code, commits to, f=
or
> > > >> example, 4.2, but then when we release 4.3, 4.4, etc they aren't o=
n
> > > >> the hook to manually tests the functionality with each release.  T=
he
> > > >> obvious nature of a community project is that people come and go.
>  If
> > > >> a contributor wants to ensure the long term viability of the
> > > >> component, they should ensure that there are unit+integration test=
s.
> > > >>
> > > >> Now, for whatever reason whether good or bad, it's not always
> possible
> > > >> to have full integration tests.  I don't want to throw down the
> gamut
> > > >> and say everything must have coverage because that will mean some
> > > >> useful code/feature will not get in because of some coverage wasn'=
t
> > > >> possible at the time.
> > > >>
> > > >> What I propose is that for every feature or function we put it in =
a
> > > >> tier of what is the quality of it (very similar to how OpenStack
> > > >> qualifies their hypervisor integration).  Tier A means unit test a=
nd
> > > >> integration test coverage gates the release.  Tier B means unit te=
st
> > > >> coverage gates the release.  Tier C mean who knows, it compiled.  =
We
> > > >> can go through and classify the components and then as a community
> we
> > > >> can try to get as much into Tier A as possible.
> > > >>
> > > >> Darren
> > > >
> > > >
> > > >
> > > > --
> > > >
> > > > EOF
> > >
> >
> >
> >
> > --
> >
> > EOF
>
> --
> Prasanna.,
>
> ------------------------
> Powered by BigRock.com
>
>


--

EOF=