incubator-cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Edison Su <>
Subject [Discuss] Integration test
Date Mon, 07 Jan 2013 22:11:03 GMT
Hi All,
     As Cloudstack is becoming more modularized, we can start thinking about how to test these
     The organization of maven projects in cloudstack. At lowest level, it's a maven project.
The modularized component, means a group of maven projects. The subsystem, means a logical
group of  components. A system, will be build on a lot of subsystems. For example, there will
be a system called, cloud-engine, which is a server, provides restful API service to other
systems(e.g API server). Cloud-engine will have a set of subsystems: computing, storage, network
etc. For each subsystem, will have a group of components: e.g. storage subsystem will include
volume/image/snapshot .. components. For each component, will have a set of maven projects,
 for example, volume component will have a maven project, called storage-volume, and a lot
of plugin projects, such as (storage-volume-plugin-solidfire, storage-volume-plugin-netapp
     Based on above hierarchy, the integration test will happen at different levels: at component
level, at subsystem level, at system level etc. I want to bring up the discuss about how to
organize integration tests, and which integration test framework should we use. 
    First, how to organize integration tests? The organization will be affected/constrained
by the dependency between maven projects you want to test.
          1. Will the integration test at each level have its own maven project?  Or we put
all the integration test cases into one giant maven project? I'd prefer one maven project
at one level, as it looks like more cleaner. For example, there will be a storage-integration-test
maven project, which depends on storage-volume-integration-test, storage-image-integration-test,
and so on, while storage-volume-integration-test will depend on storage-volume, storage-volume-plugin-solidfire,
and storage-volume-plugin-netapp and so on. When you write test case at each level, you only
focus on the limited area you want to test: the design of test case and the configuration
file of each test case, will/can only be specific to the area. Hope it will make writing test
case easier.
          2. Which test framework we should use? Junit or testNG? I was sold to testNG by
its feature sets: parametrized test case, group test cases, running test case in parallel
at different level(method, class etc), test case dependency etc. You can find out more information
about testNG at
	But in order to integrate testNG with Spring, need a little bit coding. For example, may
need to add a listener in testNG, to inject @DB annotation, and need to make sure mockito
will work. I created a base testNG class: CloudStackTestNGBase, which can read configuration
parameters from a testNG config xml file, setup @DB context for each test case etc. 

   Comments/feedback are welcome!


View raw message