cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Darren Shepherd <darren.s.sheph...@gmail.com>
Subject Re: Tiered Quality
Date Tue, 29 Oct 2013 01:21:57 GMT
Starting with the honor system might be good.  It's not so easy some times to relate lines
of code to functionality.  Also just because it hits a line of code doesn't mean it's really
tested.

Can't we just get people to just put a check mark on some table in the wiki?

Darren

> On Oct 28, 2013, at 12:08 PM, Santhosh Edukulla <santhosh.edukulla@citrix.com>
wrote:
> 
> 1.It seems we already have a code coverage numbers using sonar as below. It currently
shows only the numbers for unit tests.
> 
> https://analysis.apache.org/dashboard/index/100206
> 
> 2. The below link has an explanation for using it for both integration and unit tests.
> 
> http://docs.codehaus.org/display/SONAR/Code+Coverage+by+Integration+Tests+for+Java+Project
> 
> 3. Many links suggests it has good decision coverage facility compared to other coverage
tools.
> 
> http://onlysoftware.wordpress.com/2012/12/19/code-coverage-tools-jacoco-cobertura-emma-comparison-in-sonar/
> 
> Regards,
> Santhosh
> ________________________________________
> From: Laszlo Hornyak [laszlo.hornyak@gmail.com]
> Sent: Monday, October 28, 2013 1:43 PM
> To: dev@cloudstack.apache.org
> Subject: Re: Tiered Quality
> 
> Sonar already tracks the unit test coverage. It is also able to track the
> integration test coverage, however this might be a bit more sophisticated
> in CS since not all hardware/software requirements are available in the
> jenkins environment. However, this could be a problem in any environment.
> 
> 
>> On Mon, Oct 28, 2013 at 5:53 AM, Prasanna Santhanam <tsp@apache.org> wrote:
>> 
>> We need a way to check coverage of (unit+integration) tests. How many
>> lines of code hit on a deployed system that corresponds to the
>> component donated/committed. We don't have that for existing tests so
>> it makes it hard to judge if a feature that comes with tests covers
>> enough of itself.
>> 
>>> On Sun, Oct 27, 2013 at 11:00:46PM +0100, Laszlo Hornyak wrote:
>>> Ok, makes sense, but that sounds like even more work :) Can you share the
>>> plan on how will this work?
>>> 
>>> 
>>> On Sun, Oct 27, 2013 at 7:54 PM, Darren Shepherd <
>>> darren.s.shepherd@gmail.com> wrote:
>>> 
>>>> I think it can't be at a component level because components are too
>> large.
>>>> It needs to be at a feature for implementation level.  For example,
>> live
>>>> storage migration for xen and live storage migration for kvm (don't
>> know if
>>>> that's a real thing) would be two separate items.
>>>> 
>>>> Darren
>>>> 
>>>>> On Oct 27, 2013, at 10:57 AM, Laszlo Hornyak <
>> laszlo.hornyak@gmail.com>
>>>> wrote:
>>>>> 
>>>>> I believe this will be very useful for users.
>>>>> As far as I understand someone will have to qualify components. What
>> will
>>>>> be the method for qualification? I do not think simply the test
>> coverage
>>>>> would be right. But then if you want to go deeper, then you need a
>> bigger
>>>>> effort testing the components.
>>>>> 
>>>>> 
>>>>> 
>>>>> On Sun, Oct 27, 2013 at 4:51 PM, Darren Shepherd <
>>>>> darren.s.shepherd@gmail.com> wrote:
>>>>> 
>>>>>> I don't know if a similar thing has been talked about before but
I
>>>>>> thought I'd just throws this out there.  The ultimate way to ensure
>>>>>> quality is that we have unit test and integration test coverage on
>> all
>>>>>> functionality.  That way somebody authors some code, commits to,
for
>>>>>> example, 4.2, but then when we release 4.3, 4.4, etc they aren't
on
>>>>>> the hook to manually tests the functionality with each release. 
The
>>>>>> obvious nature of a community project is that people come and go.
>> If
>>>>>> a contributor wants to ensure the long term viability of the
>>>>>> component, they should ensure that there are unit+integration tests.
>>>>>> 
>>>>>> Now, for whatever reason whether good or bad, it's not always
>> possible
>>>>>> to have full integration tests.  I don't want to throw down the
>> gamut
>>>>>> and say everything must have coverage because that will mean some
>>>>>> useful code/feature will not get in because of some coverage wasn't
>>>>>> possible at the time.
>>>>>> 
>>>>>> What I propose is that for every feature or function we put it in
a
>>>>>> tier of what is the quality of it (very similar to how OpenStack
>>>>>> qualifies their hypervisor integration).  Tier A means unit test
and
>>>>>> integration test coverage gates the release.  Tier B means unit test
>>>>>> coverage gates the release.  Tier C mean who knows, it compiled.
 We
>>>>>> can go through and classify the components and then as a community
>> we
>>>>>> can try to get as much into Tier A as possible.
>>>>>> 
>>>>>> Darren
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> 
>>>>> EOF
>>> 
>>> 
>>> 
>>> --
>>> 
>>> EOF
>> 
>> --
>> Prasanna.,
>> 
>> ------------------------
>> Powered by BigRock.com
> 
> 
> --
> 
> EOF

Mime
View raw message