cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Edison Su <Edison...@citrix.com>
Subject RE: [DISCUSSION] CI
Date Thu, 21 Aug 2014 21:18:54 GMT


> -----Original Message-----
> From: Pierre-Luc Dion [mailto:pdion@cloudops.com]
> Sent: Thursday, August 21, 2014 11:21 AM
> To: dev@cloudstack.apache.org
> Cc: Will Stevens
> Subject: Re: [DISCUSSION] CI
> 
> On Thu, Aug 21, 2014 at 1:32 PM, Edison Su <Edison.su@citrix.com> wrote:
> 
> >
> >
> > > -----Original Message-----
> > > From: Pierre-Luc Dion [mailto:pdion@cloudops.com]
> > > Sent: Thursday, August 21, 2014 5:06 AM
> > > To: dev@cloudstack.apache.org
> > > Cc: Will Stevens
> > > Subject: Re: [DISCUSSION] CI
> > >
> > > Thanks for you responses,
> > >
> > > The revamp of marvin look promising, I'll have a closer look at it,
> > > even
> > if I'm
> > > not Python fan ;).  In our side, we have close to fully automated
> > installation
> > > of CloudStack, XenServer hypervisor and an initial zone creation.
> > > Our
> > missing
> >
> >
> > Will it be open sourced?:)
> 
> 
> Of course :-)   for the Cloudstack configuration part.
> https://github.com/cloudops/cookbook_cloudstack
> 
> For the XenServer side it's a bit arch because we basically manage xenserver
> kickstart with a Chef Cookbook that is quite ugly. but could be opensourced.

Internally in Citrix(every time I mentioned my company name in ML, I am so nerves.. ), we
almost do 
The same thing, cobber/kickstart to install hypervisors, we support KVM/Vmware/Xenserver.
Let me ask the team working it, when can we open source the code.

> 
> Zone, service offerings, templates creation, I have a set of scripts not on git
> hub yet (still too ugly) but marven does it too.
> 
> 
> 
> > > part is automated test cases to use it. So we could provide
> > infrastructure and
> > > run tests against it for one or more  test scenarios during QA phase
> > > of
> > release.
> > > Hopefully this could be integrated to the CI.
> > Currently http://jenkins.buildacloud.org/, is CloudStack CI portal.
> > If you can provide Infrastructure to integrate with it, that will be
> > awesome!
> >
> > >
> > > Would it make sense to have a sheet, maybe a wiki page that list
> > > all infrastructure test scenarios and their result per version of ACS?
> > > for:
> > > xenserver, hyperv, vmware,kvm
> > > advanced, simplenetworking, vlan/vxlan,gre isolated, networks
> > s3,swift,nfs,
> > > storage plugin
> > >
> > > Unless we have this kind of results somewhere, I don't feel we
> > > really
> > keep
> > > track of what has been tested on each release.
> > It's a good idea. Let me ask the team who wrote these test cases,
> > maybe they have answers.
> >
> > >
> > >
> > >
> > >
> > > *Pierre-Luc DION*
> > > Architecte de Solution Cloud | Cloud Solutions Architect t
> > > 855.652.5683
> > >
> > > *CloudOps* Votre partenaire infonuagique* | *Cloud Solutions Experts
> > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
> > > tw @CloudOps_
> > >
> > >
> > >
> > > On Thu, Aug 21, 2014 at 2:03 AM, Prasanna Santhanam <tsp@qubole.com>
> > > wrote:
> > >
> > > > I use rspec in Ruby these days and find it get complex to maintain
> > > > pretty quickly. It's good for dealing with REST apis and
> > > > middleware but with Marvin (and python in general) I find
> > > > infrastructure handling easier. Stack traces of BDD code is also a
> > > > little bit hard to troubleshoot and comprehend. I may be biased
> > > > because my Python expertise is better, so you can take this with a pinch
> of salt.
> > > >
> > > > Edison is right however, the setup in marvin is atrociously
> > > > complex and needs to be a lot simpler and intuitive. In most cases
> > > > I don't want to specify tons of args (service offering, disk
> > > > offering, network
> > > > type) to create a simple vM that is part of every test. To this
> > > > end I refactored Marvin to use easily defined factories (in lines
> > > > with Ruby's factory_girl) and also demo-ed some code in the
> > > > branch. Haven't had time to maintain and revamp this honestly. So
> > > > if anyone wants to pick up from there, I'm happy to help.
> > > >
> > > >
> > >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Marvin+Refact
> > > or
> > > >
> > > >
> > > >
> > > > On Thu, Aug 21, 2014 at 4:30 AM, Edison Su <Edison.su@citrix.com>
> > wrote:
> > > > >
> > > > >
> > > > >> -----Original Message-----
> > > > >> From: Pierre-Luc Dion [mailto:pdion@cloudops.com]
> > > > >> Sent: Wednesday, August 20, 2014 11:20 AM
> > > > >> To: dev@cloudstack.apache.org; Will Stevens
> > > > >> Subject: [DISCUSSION] CI
> > > > >>
> > > > >> I'd like to move this discussion out of the GIT workflow thread.
> > > > >>
> > > > >> Do we have any CI strategy in place or a starting point at least?
> > > > > TravisCI will be a starting point for now. It tests against
> > > > > simulator
> > > > only.
> > > > > I am working on a solution to test against KVM(that's most of
> > > > > users are
> > > > concerned about) in a machine I created in
> > > > > Public cloud. It has big memory, super fast disk IO, should be a
> > > > > good
> > > > solution for KVM test. It may take few days to get it work.
> > > > > They are both for simple test, and in small scale.
> > > > >>
> > > > >> I've never worked with marvin I from the small understanding
> > > > >> that I have from it, it might not be the right too. I don't know.
> > > > >>
> > > > >> I've spend some times recently with test-kitchen and serverspec
> > > > >> and
> > > > found
> > > > >> that quite promising as test platform for "real cloudstack
> > > > >> deployment
> > > > test
> > > > >> scenarios". By using test framework like Rspec the test output
> > > > >> is clean
> > > > to
> > > > >> read for human and test scenarios seams quite simple to write.
> > > > >>
> > > > >> I've started a rspec lib for cloudstack to enhance serverspec
> > > > >> for
> > > > CloudStack
> > > > >> tests (cloudstack-rspec).  I'm wondering if it would be usable
> > > > >> in
> > > > future CI
> > > > >> system for CloudStack...
> > > > > I am familiar with Specs,  like Rspec, but in scala. I agree
> > > > > with you
> > > > that the BDD style test case
> > > > > Is easier to read/write. It's good to have, but I think before
> > > > > we move
> > > > to new test framework, we should figure out the issues
> > > > > We have today in Marvin, so that we can take effort to address
> > > > > them,
> > > > either fixing Marvin, or switch to a new test framework.
> > > > > The issue I have in current Marvin, is that it still needs to
> > > > > write a
> > > > lot of setup code for each test case, which is complicated enough.
> > > > Let's take an Marvin test case as an example:
> > > > https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=
> > > > test
> > > >
> > >
> /integration/smoke/test_vm_life_cycle.py;h=57e47c24838ffc2ec5e9af4ad
> > > 38
> > > > 72d5e27342d91;hb=HEAD
> > > > >
> > > > > Like this kind of code in test case setup code:
> > > > >
> > > > > # Get Zone, Domain and templates
> > > > >         cls.domain = get_domain(cls.apiclient)
> > > > >         cls.zone = get_zone(cls.apiclient,
> > testClient.getZoneForTests())
> > > > >         cls.services['mode'] = cls.zone.networktype
> > > > >
> > > > >         #If local storage is enabled, alter the offerings to use
> > > > localstorage
> > > > >         #this step is needed for devcloud
> > > > >         if cls.zone.localstorageenabled == True:
> > > > >
> > > > > cls.services["service_offerings"]["tiny"]["storagetype"]
> > > > > =
> > > > 'local'
> > > > >
> > > > > cls.services["service_offerings"]["small"]["storagetype"] =
> > > > 'local'
> > > > >
> > > > > cls.services["service_offerings"]["medium"]["storagetype"] =
> > > > 'local'
> > > > >
> > > > > this kind of information should be provided by test framework
> > > > automatically, instead of explicitly set in every test case.
> > > > > Do you have other concerns about current Marvin, other than test
> > style?
> > > > >
> > > > >
> > > > >>
> > > > >> Also, having a CI in place for cloudstack is quite of a huge
> > > > >> deal if we
> > > > want to
> > > > >> validate all kind of deployment scenario, networks, hypervisor
> > > > >> version
> > > > and
> > > > >> guest OS version.
> > > > >
> > > > >
> > > > > Can't agree with you more.
> > > > >
> > > > >>
> > > > >>
> > > > >> For those that have some knowledge on the CI subject in
> > > > >> CloudStack
> > > > please
> > > > >> update the community. I don't feel alone being lost  about this
> > > > CloudStack
> > > > >> topic.
> > > > >
> > > > > There is an update-to-date write up regarding to Marvin:
> > > > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Marvin+-
> > > +Testin
> > > > g+with+Python
> > > > > Marvin does two things, one is to call cloudstack mgt API to add
> > > > resources into CloudStack mgt server, such as hypervisors, primary
> > > > storages, networks, and secondary storages etc.
> > > > > Another one is to run test cases under tools/integration/smoke
> > > > > or any
> > > > other python test cases.
> > > > > It doesn't cover how to deploy hypervisors on real HW, how to
> > > > > setup
> > > > primary storages, secondary storage etc. That's something like
> > > > test-kitchen and serverspec can help, I think.
> > > > >
> > > > >>
> > > > >> Thanks,
> > > > >>
> > > > >>
> > > > >> Pierre-Luc
> > > >
> > > >
> > > >
> > > > --
> > > > Prasanna.,
> > > >
> >
Mime
View raw message