cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Edison Su <Edison...@citrix.com>
Subject RE: [DISCUSS] Breaking out Marvin from CloudStack
Date Tue, 08 Oct 2013 18:22:44 GMT


> -----Original Message-----
> From: Santhosh Edukulla [mailto:santhosh.edukulla@citrix.com]
> Sent: Tuesday, October 08, 2013 1:28 AM
> To: dev@cloudstack.apache.org
> Subject: RE: [DISCUSS] Breaking out Marvin from CloudStack
> 
> Comments Inline.
> 
> -----Original Message-----
> From: Edison Su [mailto:Edison.su@citrix.com]
> Sent: Tuesday, October 08, 2013 4:18 AM
> To: dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>
> Subject: RE: [DISCUSS] Breaking out Marvin from CloudStack
> 
> Few questions:
> 1. About the "more object-oriented" CloudStack API python binding: Is the
> proposed api  good enough?
> For example,
> The current hand written create virtual machine looks like:
> class VirtualMachine(object):
> ....
>     @classmethod
>     def create(cls, apiclient, services, templateid=None, accountid=None,
>                     domainid=None, zoneid=None, networkids=None,
> serviceofferingid=None,
>                     securitygroupids=None, projectid=None, startvm=None,
>                     diskofferingid=None, affinitygroupnames=None, group=None,
>                     hostid=None, keypair=None, mode='basic', method='GET'):
> 
> the proposed api may look like:
> 
> class VirtualMachine(object):
>    def create(self, apiclient, accountId, templateId, **kwargs)
> 
> The proposed api will look better than previous one, and it's automatically
> generated, so easy to maintain. But as a consumer of the api, how do people
> know what kind of parameters should be passed in? Will you have an online
> document for your api? Or you assume people will look at the api docs
> generated by CloudStack?
> Or why not make the api itself as self-contained? For example, add docs
> before create method:
> 
> class VirtualMachine(object):
>    '''
>      Args:
>       accountId: what ever
>        templateId: whatever
>        networkids: whatever
>    '''
>    '''
>    Response:
>     '''
>    def create(self, apiclient, accountId, templateId, **kwargs)
> 
> All the api documents should be included in api discovery already, so it should
> be easy to add them in your api binding.
> >> [Santhosh]: Each verb as an action on entity, will have provision as earlier
> to have all required and as well optional arguments.  Regarding doc strings, If
> the API docs are having  this facilitation, we will  add them as corresponding
> doc strings during generation of python binding and as well entities. As you
> rightly mentioned, it will good to add this . We will make sure to get it.
> Adding adequate doc strings applies even while writing test feature\lib as
> well, it will improve  ease ,readability,usage etc. Anyways a wiki page, and
> additional pydoc documents posted online will be there.
That's great! The way you separate required parameters from optional parameters on the method
signature, is quite a good idea.

> 
> 2. Regarding to data factories. From the proposed factories, in each test case,
> does test writer still need to write the code to get data, such as writing code
> to get account during the setupclass?
> I looked at some of the existing test cases, most of them have the same code
> snippet:
> class Services:
>     def __init__(self):
>         self.services = {
>             "account": {
>                 "email": "test@test.com<mailto:test@test.com>",
>                 "firstname": "Test",
>                 "lastname": "User",
>                 "username": "test",
>                 "password": "password",
>             },
>             "virtual_machine": {
>                 "displayname": "Test VM",
>                 "username": "root",
>                 "password": "password",
>                 "ssh_port": 22,
>                 "hypervisor": 'XenServer',
>                 "privateport": 22,
>                 "publicport": 22,
>                 "protocol": 'TCP',
>             },
> 
> With the data factories, the code will look like the following?
> 
> Class TestFoo:
>      Def setupClass():
>               Account = UserAccount(apiclient)
>                VM = UserVM(apiClient)
> 
> And if I want to customize the default data factories, I should be able to use
> something like: UserAccount(apiclient, username='myfoo')?
> And the data factories should be able to customized based on test
> environment, right?
> For example, the current iso test cases are hardcoded to test against
> http://people.apache.org/~tsp/dummy.iso, but it won't work for devcloud,
> or in an internal network. The ISO data factory should be able to return an url
> based on different test environment, thus iso test cases can be reused.
> 
> >>[Santhosh] : Currently, as you mentioned, Services class is part of many
> test modules, this is basically data part for the test. We are separating this
> with factory approach. Thus, segregating data from test. Compare the earlier
> mention of Services class in earlier test code without Service class in the
> below test code.
> 
> class TestVpcLifeCycle(cloudstackTestCase):
> 
>     def setUp(self):
>         self.apiclient = super(TestVpcLifeCycle,
> self).getClsTestClient().getApiClient()
>         self.zoneid = get_zone(self.apiclient).id
>         self.templateid = get_template(self.apiclient).id
>         self.serviceofferingid = get_service_offering(self.apiclient).id
>         self.account = UserAccount( apiclient=self.apiclient ) ---> Data factory
> creation
> 
> 
>     @attr(tags='debug')
>     def test_deployvm(self):
>         vm = VpcVirtualMachine(
>             apiclient=self.apiclient,
>             account=self.account.name,
>             domainid=self.account.domainid,
>             zoneid=self.zoneid,
>             templateid=self.templateid,
>             serviceofferingid=self.serviceofferingid
>         )
>         vm.state | should | equal_to('Running')
> 
> There could be many ways of data driving\In this case providing simple data
> to the test cases. Factory approach though better is driven to create factories
> with default sequences once and reused. Yes, user can customize a data
> factory as you mentioned. customizations specific to environment are not
> available though, but doable while creating a factory instance. We can take a
> care to look at it. Basic data factory for an entity by default will have values
> based upon environment available and then user can customize accordingly,
> this can be added to proposal as edit.    But using factories logic, do you see
> any issues as a whole in long run? Any disadvantages you foresee? Or using
> plain data driving approach through sample xml\csv file for a given test
> module should suffice i.e., provide a facilitation as part of current testclient
> itself as  dynamic data object creation as similar to the one provided by db
> connection as part of test client currently, the new data object therefore as
> part of testclient will then replace the current Service Class\object creation
> for a given test module. Loading the values from xml\csv in to data object for
> test can be controlled through a config value( path to data values file(xml\csv)
> for a given test module ) and are loaded during the object creation? This
> object through testclient  is available for test module to be used as a whole?

To me, a simple data driven approach sounds suffice. For example, anything like
Get_iso_url() which will return an usable url from a json based test data configuration file,
Or get_account() which will return an default account, is much better than put the data in
the test cases itself.
Just keep in mind, whenever their general requirements in each test, put them in either a
helper library, or inject them into each test case.

> 
> Thanks!
> Santhosh
> > -----Original Message-----
> > From: Santhosh Edukulla [mailto:santhosh.edukulla@citrix.com]
> > Sent: Monday, October 07, 2013 7:06 AM
> > To: dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>
> > Subject: RE: [DISCUSS] Breaking out Marvin from CloudStack
> >
> > Team,
> >
> > Apart\Away from breaking out marvin from cloudstack, please check the
> > other new details provided as part of the new proposal for marvin
> > refactoring. Your inputs out of experience are invaluable. Any new
> > feature tests for CS will be followed with the new approach, provided
> > if we agree to all. Pasting the proposal link one more time below.
> >
> >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Marvin+Refactor
> >
> > Regards,
> > Santhosh
> > ________________________________________
> > From: Daan Hoogland [daan.hoogland@gmail.com]
> > Sent: Sunday, October 06, 2013 3:05 PM
> > To: dev
> > Subject: Re: [DISCUSS] Breaking out Marvin from CloudStack
> >
> > On Sun, Oct 6, 2013 at 8:10 PM, Animesh Chaturvedi <
> > animesh.chaturvedi@citrix.com<mailto:animesh.chaturvedi@citrix.com>>
> wrote:
> >
> > > > Yes and we will need to work down a backlog of scenarios before we
> > > > ever can rely on guys like me doing that. Not because they won't
> > > > but because there is to much to write tests for edging on the new
> > > > features they write. Just because those tests aren't there yet. I
> > > > think giving Citrix QA a repo to work on is fine but I would like
> > > > to see it merged back at some point and a continued possibility to
> > > > write
> > them in the main tree.
> > > >
> > > [Animesh>] While I don't agree to a separate repo for tests (marvin
> > > framework is ok) I do want to call out the proposal is not for
> > > giving Citrix QA a repo to work on and I don't think Prasanna meant that
> way.
> > >
> >
> >
> > I have to apologize for the formulations I choose to express my
> > thoughts with. I did not mean to talk of a department of a certain
> > company donating human effort for testing to the community. I was
> > talking of the frustration of the individuals working and how the
> > separate repo would smoothen their workflow. The new repo is in the
> > apache domain, no question whether the work in their is done by one
> person or 100 companies.


Mime
View raw message