incubator-cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Huang <Alex.Hu...@citrix.com>
Subject RE: [Discuss] Integration test
Date Tue, 08 Jan 2013 00:00:11 GMT
Marvin is testing for an integrated system.

Here, we're talking about testing an integration contract.  This means providing tests that
represents how a plugin would be used by cloudstack.  A plugin writer should be able to run
their plugin within this test suite WITHOUT cloudstack and get some guarantee that their plugin
supports the interface correctly and won't break anything.  This is purely developer level
integration test. 

This means that if someone changes the logic in the subsystem, they are also responsible for
changing the integration test cases.

Note that this type of testing cannot replace marvin.

--Alex

> -----Original Message-----
> From: Edison Su [mailto:Edison.su@citrix.com]
> Sent: Monday, January 07, 2013 3:13 PM
> To: cloudstack-dev@incubator.apache.org
> Subject: RE: [Discuss] Integration test
> 
> No, marvin is suitable for API test, can't test java maven project directly. Here,
> the integration test, is more about test case for a set of java maven projects,
> so it must be tested by using java test framework, either junit or testNG. For
> unit test, test for an individual maven project, it's ok to use Junit. But for
> integration test, I'd prefer testNG.
> 
> > -----Original Message-----
> > From: Sudha Ponnaganti [mailto:sudha.ponnaganti@citrix.com]
> > Sent: Monday, January 07, 2013 3:07 PM
> > To: cloudstack-dev@incubator.apache.org
> > Subject: RE: [Discuss] Integration test
> >
> > Edison,
> >
> > Current python framework ( marvin) covers several of the
> components/sub-
> > modules and system ( end to end) tests. Functionally it is serving the
> purpose.
> > Is this proposal to substitute that as well as current Junit framework??
> >
> > Thanks
> > /Sudha
> >
> >
> > -----Original Message-----
> > From: Edison Su [mailto:Edison.su@citrix.com]
> > Sent: Monday, January 07, 2013 2:11 PM
> > To: cloudstack-dev@incubator.apache.org
> > Subject: [Discuss] Integration test
> >
> > Hi All,
> >      As Cloudstack is becoming more modularized, we can start thinking
> about
> > how to test these components.
> >      The organization of maven projects in cloudstack. At lowest level, it's a
> > maven project. The modularized component, means a group of maven
> > projects. The subsystem, means a logical group of  components. A system,
> > will be build on a lot of subsystems. For example, there will be a system
> > called, cloud-engine, which is a server, provides restful API service to other
> > systems(e.g API server). Cloud-engine will have a set of subsystems:
> > computing, storage, network etc. For each subsystem, will have a group of
> > components: e.g. storage subsystem will include volume/image/snapshot ..
> > components. For each component, will have a set of maven projects,  for
> > example, volume component will have a maven project, called storage-
> > volume, and a lot of plugin projects, such as (storage-volume-plugin-
> solidfire,
> > storage-volume-plugin-netapp etc).
> >      Based on above hierarchy, the integration test will happen at different
> > levels: at component level, at subsystem level, at system level etc. I want
> to
> > bring up the discuss about how to organize integration tests, and which
> > integration test framework should we use.
> >     First, how to organize integration tests? The organization will be
> > affected/constrained by the dependency between maven projects you
> want
> > to test.
> >           1. Will the integration test at each level have its own maven project?
> Or
> > we put all the integration test cases into one giant maven project? I'd
> prefer
> > one maven project at one level, as it looks like more cleaner. For example,
> > there will be a storage-integration-test maven project, which depends on
> > storage-volume-integration-test, storage-image-integration-test, and so
> on,
> > while storage-volume-integration-test will depend on storage-volume,
> > storage-volume-plugin-solidfire, and storage-volume-plugin-netapp and so
> > on. When you write test case at each level, you only focus on the limited
> area
> > you want to test: the design of test case and the configuration file of each
> > test case, will/can only be specific to the area. Hope it will make writing test
> > case easier.
> >           2. Which test framework we should use? Junit or testNG? I was sold to
> > testNG by its feature sets: parametrized test case, group test cases,
> running
> > test case in parallel at different level(method, class etc), test case
> > dependency etc. You can find out more information about testNG at
> > http://kaczanowscy.pl/tomek/sites/default/files/testng_vs_junit.txt.slidy_.
> > html#(1).
> > 	But in order to integrate testNG with Spring, need a little bit coding.
> > For example, may need to add a listener in testNG, to inject @DB
> annotation,
> > and need to make sure mockito will work. I created a base testNG class:
> > CloudStackTestNGBase, which can read configuration parameters from a
> > testNG config xml file, setup @DB context for each test case etc.
> >
> >    Comments/feedback are welcome!
> >
> >

Mime
View raw message