Mailing-List: contact cloudstack-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: cloudstack-dev@incubator.apache.org
Received-SPF: pass (nike.apache.org: domain of Prasanna.Santhanam@citrix.com
 designates 203.166.19.134 as permitted sender)
Date: Thu, 7 Jun 2012 17:41:15 +0530
From: Prasanna Santhanam <prasanna.santhanam@citrix.com>
To: "cloudstack-dev@incubator.apache.org"
	<cloudstack-dev@incubator.apache.org>
Subject: Re: bvt question
Message-ID: <20120607121115.GJ16078@cloud.com>
Mail-Followup-To: "cloudstack-dev@incubator.apache.org"
 <cloudstack-dev@incubator.apache.org>
References: 
 <CAK1eeKwtiZnWKD2W_uFiE6-4fd7semQezP51iCfKqxgA+E3V8Q@mail.gmail.com>
 <20120606084302.GC13905@cloud.com>
 <C66C814C1ABFA8449FF65CB44953A8B4011A43292634@SJCPMAILBOX01.citrite.net>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: 
 <C66C814C1ABFA8449FF65CB44953A8B4011A43292634@SJCPMAILBOX01.citrite.net>
User-Agent: Mutt/1.5.21 (2010-09-15)

On Wed, Jun 06, 2012 at 02:11:51PM -0400, Edison Su wrote:
> > For 2> I'm not sure how to get that going. The test code right now is
> > free to use any library to perform these kind of backend checks. So it
> > would be harder to have these tests skipped with a simple framework
> > fix.
> 
> How about add another annotation on test case, such as
> @hypervisor("all-except-simulator"), means this specific test case
> can only be executed when the zone has real hypervisors?

That would be nice to have. But then more than 50% of the tests
perform some or the other form of backend verification. So the decorator
will run on too few tests. I've startd porting the tests to work on the
simulator so we can have a small jenkins job to start with that runs
against each checkin to begin with. 

Another idea that Chirag (@clogeny) discussed with me was adding tags
to test cases. So a test_port_forward_vm would be tagged with a list
of attributes like:

def test_port_forward_vm(cloudstackTestCase):
	...
	...
	tags = ['network','port forward', 'advanced', 'xenserver']

This way when I want to run all network tests I can identify and run
those. And then if I wanted to run xenserver tests I can do the same.
Or if it's only going to be an advanced zone I can filter those too.

What do you think?


-- 
Prasanna.,