Mailing-List: contact cloudstack-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: cloudstack-dev@incubator.apache.org
Received-SPF: pass (athena.apache.org: domain of Edison.su@citrix.com
 designates 66.165.176.63 as permitted sender)
From: Edison Su <Edison.su@citrix.com>
To: "cloudstack-dev@incubator.apache.org"
	<cloudstack-dev@incubator.apache.org>
Date: Mon, 7 Jan 2013 15:13:14 -0800
Subject: RE: [Discuss] Integration test 
Thread-Topic: [Discuss] Integration test 
Thread-Index: Ac3tI6eaDLl2wJRfTOu/b0tW59IetwAB5DMQAAAwi4A=
Message-ID: 
 <C66C814C1ABFA8449FF65CB44953A8B4012CBF3A573F@SJCPMAILBOX01.citrite.net>
References: 
 <C66C814C1ABFA8449FF65CB44953A8B4012CBF3A573E@SJCPMAILBOX01.citrite.net>
 <7914B38A4445B34AA16EB9F1352942F1012F1461F817@SJCPMAILBOX01.citrite.net>
In-Reply-To: 
 <7914B38A4445B34AA16EB9F1352942F1012F1461F817@SJCPMAILBOX01.citrite.net>
Accept-Language: en-US
Content-Language: en-US
acceptlanguage: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

No, marvin is suitable for API test, can't test java maven project directly=
. Here, the integration test, is more about test case for a set of java mav=
en projects, so it must be tested by using java test framework, either juni=
t or testNG. For unit test, test for an individual maven project, it's ok t=
o use Junit. But for integration test, I'd prefer testNG.=20

> -----Original Message-----
> From: Sudha Ponnaganti [mailto:sudha.ponnaganti@citrix.com]
> Sent: Monday, January 07, 2013 3:07 PM
> To: cloudstack-dev@incubator.apache.org
> Subject: RE: [Discuss] Integration test
>=20
> Edison,
>=20
> Current python framework ( marvin) covers several of the components/sub-
> modules and system ( end to end) tests. Functionally it is serving the pu=
rpose.
> Is this proposal to substitute that as well as current Junit framework??
>=20
> Thanks
> /Sudha
>=20
>=20
> -----Original Message-----
> From: Edison Su [mailto:Edison.su@citrix.com]
> Sent: Monday, January 07, 2013 2:11 PM
> To: cloudstack-dev@incubator.apache.org
> Subject: [Discuss] Integration test
>=20
> Hi All,
>      As Cloudstack is becoming more modularized, we can start thinking ab=
out
> how to test these components.
>      The organization of maven projects in cloudstack. At lowest level, i=
t's a
> maven project. The modularized component, means a group of maven
> projects. The subsystem, means a logical group of  components. A system,
> will be build on a lot of subsystems. For example, there will be a system
> called, cloud-engine, which is a server, provides restful API service to =
other
> systems(e.g API server). Cloud-engine will have a set of subsystems:
> computing, storage, network etc. For each subsystem, will have a group of
> components: e.g. storage subsystem will include volume/image/snapshot ..
> components. For each component, will have a set of maven projects,  for
> example, volume component will have a maven project, called storage-
> volume, and a lot of plugin projects, such as (storage-volume-plugin-soli=
dfire,
> storage-volume-plugin-netapp etc).
>      Based on above hierarchy, the integration test will happen at differ=
ent
> levels: at component level, at subsystem level, at system level etc. I wa=
nt to
> bring up the discuss about how to organize integration tests, and which
> integration test framework should we use.
>     First, how to organize integration tests? The organization will be
> affected/constrained by the dependency between maven projects you want
> to test.
>           1. Will the integration test at each level have its own maven p=
roject?  Or
> we put all the integration test cases into one giant maven project? I'd p=
refer
> one maven project at one level, as it looks like more cleaner. For exampl=
e,
> there will be a storage-integration-test maven project, which depends on
> storage-volume-integration-test, storage-image-integration-test, and so o=
n,
> while storage-volume-integration-test will depend on storage-volume,
> storage-volume-plugin-solidfire, and storage-volume-plugin-netapp and so
> on. When you write test case at each level, you only focus on the limited=
 area
> you want to test: the design of test case and the configuration file of e=
ach
> test case, will/can only be specific to the area. Hope it will make writi=
ng test
> case easier.
>           2. Which test framework we should use? Junit or testNG? I was s=
old to
> testNG by its feature sets: parametrized test case, group test cases, run=
ning
> test case in parallel at different level(method, class etc), test case
> dependency etc. You can find out more information about testNG at
> http://kaczanowscy.pl/tomek/sites/default/files/testng_vs_junit.txt.slidy=
_.
> html#(1).
> 	But in order to integrate testNG with Spring, need a little bit coding.
> For example, may need to add a listener in testNG, to inject @DB annotati=
on,
> and need to make sure mockito will work. I created a base testNG class:
> CloudStackTestNGBase, which can read configuration parameters from a
> testNG config xml file, setup @DB context for each test case etc.
>=20
>    Comments/feedback are welcome!
>=20
>=20