cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Prasanna Santhanam <...@qubole.com>
Subject Re: [DISCUSSION] CI
Date Thu, 21 Aug 2014 06:03:12 GMT
I use rspec in Ruby these days and find it get complex to maintain
pretty quickly. It's good for dealing with REST apis and middleware
but with Marvin (and python in general) I find infrastructure handling
easier. Stack traces of BDD code is also a little bit hard to
troubleshoot and comprehend. I may be biased because my Python
expertise is better, so you can take this with a pinch of salt.

Edison is right however, the setup in marvin is atrociously complex
and needs to be a lot simpler and intuitive. In most cases I don't
want to specify tons of args (service offering, disk offering, network
type) to create a simple vM that is part of every test. To this end I
refactored Marvin to use easily defined factories (in lines with
Ruby's factory_girl) and also demo-ed some code in the branch. Haven't
had time to maintain and revamp this honestly. So if anyone wants to
pick up from there, I'm happy to help.

https://cwiki.apache.org/confluence/display/CLOUDSTACK/Marvin+Refactor



On Thu, Aug 21, 2014 at 4:30 AM, Edison Su <Edison.su@citrix.com> wrote:
>
>
>> -----Original Message-----
>> From: Pierre-Luc Dion [mailto:pdion@cloudops.com]
>> Sent: Wednesday, August 20, 2014 11:20 AM
>> To: dev@cloudstack.apache.org; Will Stevens
>> Subject: [DISCUSSION] CI
>>
>> I'd like to move this discussion out of the GIT workflow thread.
>>
>> Do we have any CI strategy in place or a starting point at least?
> TravisCI will be a starting point for now. It tests against simulator only.
> I am working on a solution to test against KVM(that's most of users are concerned about)
in a machine I created in
> Public cloud. It has big memory, super fast disk IO, should be a good solution for KVM
test. It may take few days to get it work.
> They are both for simple test, and in small scale.
>>
>> I've never worked with marvin I from the small understanding that I have
>> from it, it might not be the right too. I don't know.
>>
>> I've spend some times recently with test-kitchen and serverspec and found
>> that quite promising as test platform for "real cloudstack deployment  test
>> scenarios". By using test framework like Rspec the test output is clean to
>> read for human and test scenarios seams quite simple to write.
>>
>> I've started a rspec lib for cloudstack to enhance serverspec for CloudStack
>> tests (cloudstack-rspec).  I'm wondering if it would be usable in future CI
>> system for CloudStack...
> I am familiar with Specs,  like Rspec, but in scala. I agree with you that the BDD style
test case
> Is easier to read/write. It's good to have, but I think before we move to new test framework,
we should figure out the issues
> We have today in Marvin, so that we can take effort to address them, either fixing Marvin,
or switch to a new test framework.
> The issue I have in current Marvin, is that it still needs to write a lot of setup code
for each test case, which is complicated enough. Let's take an Marvin test case as an example:
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=test/integration/smoke/test_vm_life_cycle.py;h=57e47c24838ffc2ec5e9af4ad3872d5e27342d91;hb=HEAD
>
> Like this kind of code in test case setup code:
>
> # Get Zone, Domain and templates
>         cls.domain = get_domain(cls.apiclient)
>         cls.zone = get_zone(cls.apiclient, testClient.getZoneForTests())
>         cls.services['mode'] = cls.zone.networktype
>
>         #If local storage is enabled, alter the offerings to use localstorage
>         #this step is needed for devcloud
>         if cls.zone.localstorageenabled == True:
>             cls.services["service_offerings"]["tiny"]["storagetype"] = 'local'
>             cls.services["service_offerings"]["small"]["storagetype"] = 'local'
>             cls.services["service_offerings"]["medium"]["storagetype"] = 'local'
>
> this kind of information should be provided by test framework automatically, instead
of explicitly set in every test case.
> Do you have other concerns about current Marvin, other than test style?
>
>
>>
>> Also, having a CI in place for cloudstack is quite of a huge deal if we want to
>> validate all kind of deployment scenario, networks, hypervisor version and
>> guest OS version.
>
>
> Can't agree with you more.
>
>>
>>
>> For those that have some knowledge on the CI subject in CloudStack please
>> update the community. I don't feel alone being lost  about this CloudStack
>> topic.
>
> There is an update-to-date write up regarding to Marvin: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Marvin+-+Testing+with+Python
> Marvin does two things, one is to call cloudstack mgt API to add resources into CloudStack
mgt server, such as hypervisors, primary storages, networks, and secondary storages etc.
> Another one is to run test cases under tools/integration/smoke or any other python test
cases.
> It doesn't cover how to deploy hypervisors on real HW, how to setup primary storages,
secondary storage etc. That's something like test-kitchen and serverspec can help, I think.
>
>>
>> Thanks,
>>
>>
>> Pierre-Luc



-- 
Prasanna.,

Mime
View raw message