cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pierre-Luc Dion <pd...@cloudops.com>
Subject Re: [DISCUSSION] CI
Date Thu, 21 Aug 2014 18:20:33 GMT
On Thu, Aug 21, 2014 at 1:32 PM, Edison Su <Edison.su@citrix.com> wrote:

>
>
> > -----Original Message-----
> > From: Pierre-Luc Dion [mailto:pdion@cloudops.com]
> > Sent: Thursday, August 21, 2014 5:06 AM
> > To: dev@cloudstack.apache.org
> > Cc: Will Stevens
> > Subject: Re: [DISCUSSION] CI
> >
> > Thanks for you responses,
> >
> > The revamp of marvin look promising, I'll have a closer look at it, even
> if I'm
> > not Python fan ;).  In our side, we have close to fully automated
> installation
> > of CloudStack, XenServer hypervisor and an initial zone creation.  Our
> missing
>
>
> Will it be open sourced?:)


Of course :-)   for the Cloudstack configuration part.
https://github.com/cloudops/cookbook_cloudstack

For the XenServer side it's a bit arch because we basically manage
xenserver kickstart with a Chef Cookbook that is quite ugly. but could be
opensourced.

Zone, service offerings, templates creation, I have a set of scripts not on
git hub yet (still too ugly) but marven does it too.



> > part is automated test cases to use it. So we could provide
> infrastructure and
> > run tests against it for one or more  test scenarios during QA phase of
> release.
> > Hopefully this could be integrated to the CI.
> Currently http://jenkins.buildacloud.org/, is CloudStack CI portal.  If
> you can provide
> Infrastructure to integrate with it, that will be awesome!
>
> >
> > Would it make sense to have a sheet, maybe a wiki page that list  all
> > infrastructure test scenarios and their result per version of ACS?
> > for:
> > xenserver, hyperv, vmware,kvm
> > advanced, simplenetworking, vlan/vxlan,gre isolated, networks
> s3,swift,nfs,
> > storage plugin
> >
> > Unless we have this kind of results somewhere, I don't feel we really
> keep
> > track of what has been tested on each release.
> It's a good idea. Let me ask the team who wrote these test cases, maybe
> they have answers.
>
> >
> >
> >
> >
> > *Pierre-Luc DION*
> > Architecte de Solution Cloud | Cloud Solutions Architect t 855.652.5683
> >
> > *CloudOps* Votre partenaire infonuagique* | *Cloud Solutions Experts
> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> > @CloudOps_
> >
> >
> >
> > On Thu, Aug 21, 2014 at 2:03 AM, Prasanna Santhanam <tsp@qubole.com>
> > wrote:
> >
> > > I use rspec in Ruby these days and find it get complex to maintain
> > > pretty quickly. It's good for dealing with REST apis and middleware
> > > but with Marvin (and python in general) I find infrastructure handling
> > > easier. Stack traces of BDD code is also a little bit hard to
> > > troubleshoot and comprehend. I may be biased because my Python
> > > expertise is better, so you can take this with a pinch of salt.
> > >
> > > Edison is right however, the setup in marvin is atrociously complex
> > > and needs to be a lot simpler and intuitive. In most cases I don't
> > > want to specify tons of args (service offering, disk offering, network
> > > type) to create a simple vM that is part of every test. To this end I
> > > refactored Marvin to use easily defined factories (in lines with
> > > Ruby's factory_girl) and also demo-ed some code in the branch. Haven't
> > > had time to maintain and revamp this honestly. So if anyone wants to
> > > pick up from there, I'm happy to help.
> > >
> > >
> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Marvin+Refactor
> > >
> > >
> > >
> > > On Thu, Aug 21, 2014 at 4:30 AM, Edison Su <Edison.su@citrix.com>
> wrote:
> > > >
> > > >
> > > >> -----Original Message-----
> > > >> From: Pierre-Luc Dion [mailto:pdion@cloudops.com]
> > > >> Sent: Wednesday, August 20, 2014 11:20 AM
> > > >> To: dev@cloudstack.apache.org; Will Stevens
> > > >> Subject: [DISCUSSION] CI
> > > >>
> > > >> I'd like to move this discussion out of the GIT workflow thread.
> > > >>
> > > >> Do we have any CI strategy in place or a starting point at least?
> > > > TravisCI will be a starting point for now. It tests against
> > > > simulator
> > > only.
> > > > I am working on a solution to test against KVM(that's most of users
> > > > are
> > > concerned about) in a machine I created in
> > > > Public cloud. It has big memory, super fast disk IO, should be a
> > > > good
> > > solution for KVM test. It may take few days to get it work.
> > > > They are both for simple test, and in small scale.
> > > >>
> > > >> I've never worked with marvin I from the small understanding that
I
> > > >> have from it, it might not be the right too. I don't know.
> > > >>
> > > >> I've spend some times recently with test-kitchen and serverspec and
> > > found
> > > >> that quite promising as test platform for "real cloudstack
> > > >> deployment
> > > test
> > > >> scenarios". By using test framework like Rspec the test output is
> > > >> clean
> > > to
> > > >> read for human and test scenarios seams quite simple to write.
> > > >>
> > > >> I've started a rspec lib for cloudstack to enhance serverspec for
> > > CloudStack
> > > >> tests (cloudstack-rspec).  I'm wondering if it would be usable in
> > > future CI
> > > >> system for CloudStack...
> > > > I am familiar with Specs,  like Rspec, but in scala. I agree with
> > > > you
> > > that the BDD style test case
> > > > Is easier to read/write. It's good to have, but I think before we
> > > > move
> > > to new test framework, we should figure out the issues
> > > > We have today in Marvin, so that we can take effort to address them,
> > > either fixing Marvin, or switch to a new test framework.
> > > > The issue I have in current Marvin, is that it still needs to write
> > > > a
> > > lot of setup code for each test case, which is complicated enough.
> > > Let's take an Marvin test case as an example:
> > > https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=test
> > >
> > /integration/smoke/test_vm_life_cycle.py;h=57e47c24838ffc2ec5e9af4ad38
> > > 72d5e27342d91;hb=HEAD
> > > >
> > > > Like this kind of code in test case setup code:
> > > >
> > > > # Get Zone, Domain and templates
> > > >         cls.domain = get_domain(cls.apiclient)
> > > >         cls.zone = get_zone(cls.apiclient,
> testClient.getZoneForTests())
> > > >         cls.services['mode'] = cls.zone.networktype
> > > >
> > > >         #If local storage is enabled, alter the offerings to use
> > > localstorage
> > > >         #this step is needed for devcloud
> > > >         if cls.zone.localstorageenabled == True:
> > > >             cls.services["service_offerings"]["tiny"]["storagetype"]
> > > > =
> > > 'local'
> > > >
> > > > cls.services["service_offerings"]["small"]["storagetype"] =
> > > 'local'
> > > >
> > > > cls.services["service_offerings"]["medium"]["storagetype"] =
> > > 'local'
> > > >
> > > > this kind of information should be provided by test framework
> > > automatically, instead of explicitly set in every test case.
> > > > Do you have other concerns about current Marvin, other than test
> style?
> > > >
> > > >
> > > >>
> > > >> Also, having a CI in place for cloudstack is quite of a huge deal
> > > >> if we
> > > want to
> > > >> validate all kind of deployment scenario, networks, hypervisor
> > > >> version
> > > and
> > > >> guest OS version.
> > > >
> > > >
> > > > Can't agree with you more.
> > > >
> > > >>
> > > >>
> > > >> For those that have some knowledge on the CI subject in CloudStack
> > > please
> > > >> update the community. I don't feel alone being lost  about this
> > > CloudStack
> > > >> topic.
> > > >
> > > > There is an update-to-date write up regarding to Marvin:
> > > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Marvin+-
> > +Testin
> > > g+with+Python
> > > > Marvin does two things, one is to call cloudstack mgt API to add
> > > resources into CloudStack mgt server, such as hypervisors, primary
> > > storages, networks, and secondary storages etc.
> > > > Another one is to run test cases under tools/integration/smoke or
> > > > any
> > > other python test cases.
> > > > It doesn't cover how to deploy hypervisors on real HW, how to setup
> > > primary storages, secondary storage etc. That's something like
> > > test-kitchen and serverspec can help, I think.
> > > >
> > > >>
> > > >> Thanks,
> > > >>
> > > >>
> > > >> Pierre-Luc
> > >
> > >
> > >
> > > --
> > > Prasanna.,
> > >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message