cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Santhosh Edukulla <santhosh.eduku...@citrix.com>
Subject RE: [DISCUSS] Breaking out Marvin from CloudStack
Date Tue, 08 Oct 2013 08:27:52 GMT
Comments Inline.

-----Original Message-----
From: Edison Su [mailto:Edison.su@citrix.com]
Sent: Tuesday, October 08, 2013 4:18 AM
To: dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>
Subject: RE: [DISCUSS] Breaking out Marvin from CloudStack

Few questions:
1. About the "more object-oriented" CloudStack API python binding: Is the proposed api  good
enough?
For example,
The current hand written create virtual machine looks like:
class VirtualMachine(object):
....
    @classmethod
    def create(cls, apiclient, services, templateid=None, accountid=None,
                    domainid=None, zoneid=None, networkids=None, serviceofferingid=None,
                    securitygroupids=None, projectid=None, startvm=None,
                    diskofferingid=None, affinitygroupnames=None, group=None,
                    hostid=None, keypair=None, mode='basic', method='GET'):

the proposed api may look like:

class VirtualMachine(object):
   def create(self, apiclient, accountId, templateId, **kwargs)

The proposed api will look better than previous one, and it's automatically generated, so
easy to maintain. But as a consumer of the api, how do people know what kind of parameters
should be passed in? Will you have an online document for your api? Or you assume people will
look at the api docs generated by CloudStack?
Or why not make the api itself as self-contained? For example, add docs before create method:

class VirtualMachine(object):
   '''
     Args:
      accountId: what ever
       templateId: whatever
       networkids: whatever
   '''
   '''
   Response:
    '''
   def create(self, apiclient, accountId, templateId, **kwargs)

All the api documents should be included in api discovery already, so it should be easy to
add them in your api binding.
>> [Santhosh]: Each verb as an action on entity, will have provision as earlier to have
all required and as well optional arguments.  Regarding doc strings, If the API docs are having
 this facilitation, we will  add them as corresponding doc strings during generation of python
binding and as well entities. As you rightly mentioned, it will good to add this . We will
make sure to get it.  Adding adequate doc strings applies even while writing test feature\lib
as well, it will improve  ease ,readability,usage etc. Anyways a wiki page, and additional
pydoc documents posted online will be there.

2. Regarding to data factories. From the proposed factories, in each test case, does test
writer still need to write the code to get data, such as writing code to get account during
the setupclass?
I looked at some of the existing test cases, most of them have the same code snippet:
class Services:
    def __init__(self):
        self.services = {
            "account": {
                "email": "test@test.com<mailto:test@test.com>",
                "firstname": "Test",
                "lastname": "User",
                "username": "test",
                "password": "password",
            },
            "virtual_machine": {
                "displayname": "Test VM",
                "username": "root",
                "password": "password",
                "ssh_port": 22,
                "hypervisor": 'XenServer',
                "privateport": 22,
                "publicport": 22,
                "protocol": 'TCP',
            },

With the data factories, the code will look like the following?

Class TestFoo:
     Def setupClass():
              Account = UserAccount(apiclient)
               VM = UserVM(apiClient)

And if I want to customize the default data factories, I should be able to use something like:
UserAccount(apiclient, username='myfoo')?
And the data factories should be able to customized based on test environment, right?
For example, the current iso test cases are hardcoded to test against http://people.apache.org/~tsp/dummy.iso,
but it won't work for devcloud, or in an internal network. The ISO data factory should be
able to return an url based on different test environment, thus iso test cases can be reused.

>>[Santhosh] : Currently, as you mentioned, Services class is part of many test modules,
this is basically data part for the test. We are separating this with factory approach. Thus,
segregating data from test. Compare the earlier mention of Services class in earlier test
code without Service class in the below test code.

class TestVpcLifeCycle(cloudstackTestCase):

    def setUp(self):
        self.apiclient = super(TestVpcLifeCycle, self).getClsTestClient().getApiClient()
        self.zoneid = get_zone(self.apiclient).id
        self.templateid = get_template(self.apiclient).id
        self.serviceofferingid = get_service_offering(self.apiclient).id
        self.account = UserAccount( apiclient=self.apiclient ) ---> Data factory creation


    @attr(tags='debug')
    def test_deployvm(self):
        vm = VpcVirtualMachine(
            apiclient=self.apiclient,
            account=self.account.name,
            domainid=self.account.domainid,
            zoneid=self.zoneid,
            templateid=self.templateid,
            serviceofferingid=self.serviceofferingid
        )
        vm.state | should | equal_to('Running')

There could be many ways of data driving\In this case providing simple data to the test cases.
Factory approach though better is driven to create factories with default sequences once and
reused. Yes, user can customize a data factory as you mentioned. customizations specific to
environment are not available though, but doable while creating a factory instance. We can
take a care to look at it. Basic data factory for an entity by default will have values based
upon environment available and then user can customize accordingly, this can be added to proposal
as edit.    But using factories logic, do you see any issues as a whole in long run? Any disadvantages
you foresee? Or using plain data driving approach through sample xml\csv file for a given
test module should suffice i.e., provide a facilitation as part of current testclient itself
as  dynamic data object creation as similar to the one provided by db connection as part of
test client currently, the new data object therefore as part of testclient will then replace
the current Service Class\object creation for a given test module. Loading the values from
xml\csv in to data object for test can be controlled through a config value( path to data
values file(xml\csv) for a given test module ) and are loaded during the object creation?
This object through testclient  is available for test module to be used as a whole?

Thanks!
Santhosh
> -----Original Message-----
> From: Santhosh Edukulla [mailto:santhosh.edukulla@citrix.com]
> Sent: Monday, October 07, 2013 7:06 AM
> To: dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>
> Subject: RE: [DISCUSS] Breaking out Marvin from CloudStack
>
> Team,
>
> Apart\Away from breaking out marvin from cloudstack, please check the
> other new details provided as part of the new proposal for marvin
> refactoring. Your inputs out of experience are invaluable. Any new
> feature tests for CS will be followed with the new approach, provided
> if we agree to all. Pasting the proposal link one more time below.
>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Marvin+Refactor
>
> Regards,
> Santhosh
> ________________________________________
> From: Daan Hoogland [daan.hoogland@gmail.com]
> Sent: Sunday, October 06, 2013 3:05 PM
> To: dev
> Subject: Re: [DISCUSS] Breaking out Marvin from CloudStack
>
> On Sun, Oct 6, 2013 at 8:10 PM, Animesh Chaturvedi <
> animesh.chaturvedi@citrix.com<mailto:animesh.chaturvedi@citrix.com>> wrote:
>
> > > Yes and we will need to work down a backlog of scenarios before we
> > > ever can rely on guys like me doing that. Not because they won't
> > > but because there is to much to write tests for edging on the new
> > > features they write. Just because those tests aren't there yet. I
> > > think giving Citrix QA a repo to work on is fine but I would like
> > > to see it merged back at some point and a continued possibility to
> > > write
> them in the main tree.
> > >
> > [Animesh>] While I don't agree to a separate repo for tests (marvin
> > framework is ok) I do want to call out the proposal is not for
> > giving Citrix QA a repo to work on and I don't think Prasanna meant that way.
> >
>
>
> I have to apologize for the formulations I choose to express my
> thoughts with. I did not mean to talk of a department of a certain
> company donating human effort for testing to the community. I was
> talking of the frustration of the individuals working and how the
> separate repo would smoothen their workflow. The new repo is in the
> apache domain, no question whether the work in their is done by one person or 100 companies.


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message