cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Frank Zhang <Frank.Zh...@citrix.com>
Subject RE: [PROPOSAL] BVT for CloudStack checkins
Date Thu, 07 Mar 2013 18:49:51 GMT


> -----Original Message-----
> From: prasanna [mailto:srivatsav.prasanna@gmail.com] On Behalf Of
> Prasanna Santhanam
> Sent: Thursday, March 07, 2013 2:53 AM
> To: cloudstack-dev@incubator.apache.org
> Subject: Re: [PROPOSAL] BVT for CloudStack checkins
> 
> On Thu, Mar 07, 2013 at 01:19:19AM +0530, Frank Zhang wrote:
> > > - improve simulator for runtime testability
> > > - customize to model and inject failures
> > > - make a habit of writing tests around bug reports (I started
> > > tagging tests since api_refactoring on JIRA already.
> > > look for the integration-test label)
> > > - make integration testing easier using factories and DSLs (from
> > >   Chiradeep)
> >
> > Sorry for long  thread again. All my opinion is "don't distract from
> > tools, improving Marvin should be good enough". Below are points that
> > you can simply skip.
> 
> Frank, no need to apologize, these are useful insights. My original proposal
> was only to refactor the marvin library as you suggest. It still is to provide a
> good library that involves less typing in python.
> 
> 
> > 2. capability
> > DSL varying from common language usually lacks loop, condition
> > statement, and sometimes variable. But these things are very important
> > to our test. if you strip these abilities from your DSL, I believe you
> > will finally add them back that ends up with writing a common
> > language, then you have been distracted much from writing test case
> >
> DSL is a good to have for non-programmers - the target audience could be
> those who are not comfortable writing python code and qa engineers who
> aren't experienced writing code. Agree that it will have its limitations. The dsl
> is only for quickly writing a few simple tests, it's not to replace the actual
> integrated test writing using python. I won't be doing that as it will convolute
> things.
> 
> 
> > 3. it's not our business now
> > The urgent need for CloudStack now is a set of reliable test cases,
> > which I think could be done by improving current Marvin. Building our
> > DSL distracts us at this point.
> 
> If not for writing tests I"m sure it can be used for describing deployments for
> marvin. I'll figure out a compromise and your inputs on this is most welcome!

Cool. We used to have that but have been obsoleted.
This is most valuable thing I see for CloudStack, marvin should use XML to deploy
setup. This may be the most widely spread DSL that is comfortable for even
non-programmer 

> 
> > 4. you don't even need to search a test framework for this time being
> > When I studying DSL, I ever looked into some famous test framework.
> > For Behavior Driven Test I looked Cucumber(Ruby), for Model Driven
> > Test I looked fpmt(the name maybe wrong, written by a French). They
> > both use DSL. My conclusion is the benefit of DSL is for stuff that
> > has nothing to do with test logic itself. For example, driving test
> > cases, producing document/report, scheduling random test case
> > combination ...
> 
> I did read enough dsl-bashing on the interwebs when I explored about it
> (terms like cargo cults from the ruby era come to mind) but it might be worth
> a shot. We won't know until we try.
> 
> > they are nice to have, but not urgent. So for our purpose, I would
> > suggest using  python nose which is a unit test case framework but can
> > still use for integration test. simple, quick, and easy.
> 
> The nose support for marvin has existed for sometime and you can use many
> of nose's plugins today including - pdb, attributes, multiprocess etc.
> 
> --
> Prasanna.,

Mime
View raw message