cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Nalley <da...@gnsa.us>
Subject Re: [DISCUSS] State of 4.5 and testing in local environments
Date Thu, 30 Apr 2015 11:21:02 GMT
Just as an FYI, we've dramatically increased Travis capacity.

https://blogs.apache.org/infra/entry/apache_gains_additional_travis_ci

On Thursday, April 30, 2015, Abhinandan Prateek <
abhinandan.prateek@shapeblue.com> wrote:

> Yes, lot of sanity testing can be accomplished by the automated test
> suites that we already have.
>
> There are still lot of tests that can be run using the travis-CI, (need to
> see how many more without loading the travis system). Some of the companies
> including Shapeblue are already running most or all of the test cases.
> This means that the latest branches  (4.5, 4.6) already have a level of
> sanity.
>
> At this stage we should define the amount of coverage that automation
> provides and also look at the areas where automation is failing, and work
> towards covering it.  We need this as we continue to find issues that are
> not covered by automation.
>
> I agree that while releasing if issues are found then we fix them and also
> write tests so that these do not reoccur.
>
> -abhi
>
> > On 30-Apr-2015, at 7:42 am, David Nalley <david@gnsa.us <javascript:;>>
> wrote:
> >
> > The problem is really two fold.
> > Cutting an RC isn't terribly time consuming, once you've done it a
> > dozen times or so, it goes by pretty quickly.
> >
> > The problem is the time it takes to do any material testing; and that
> > we don't have a way of telling what the status of a candidate is
> > without doing lots of manual testing. We've had multiple cases of
> > having enough binding votes for 4.5.x candidates only for someone to
> > find problems. This leads me (and this could just be my perspective)
> > to let an RC sit for a while - get folks to test it. My experience as
> > a person voting is that rapid churn of RCs leads me to not even
> > attempt to setup a testing environment, much less do any testing. I
> > found that by the time I could get around to testing an RC vote had
> > been cancelled and it had been rerolled, sometimes twice.
> >
> > I've said this before, and I'll toss it out again. I think we have to
> > start 'trusting' the automated tests. We find lots of problems
> > manually, but we don't improve the testing situation for those blocker
> > issues that stopped us from releasing. I'm of the opinion we should
> > release quickly, very quickly. But if we have to spend a week just
> > looking for bugs, that will never fly. So - let the standard be
> > automated tests - and if there's a problem, we create an automated
> > test for it, and solve in a week or two. That will force us to live
> > and die by automated tests.
> >
> > --David
> >
> > On Wed, Apr 29, 2015 at 9:45 AM, Rohit Yadav <rohit.yadav@shapeblue.com
> <javascript:;>> wrote:
> >> Hi Wilder - Thanks we’ll need all the ammo we have :) I’ve already done
> my testings wrt XS 6.2/6.5 and KVM qemu 2.0/2.1/2.2 so will test KVM 1.5
> and VMware 5.5.
> >>
> >> Hi Daan - makes sense, we’re “so" late with 4.5.1 that we need to focus
> on 4.6.0 to avoid delaying it. I’ve asked David regarding co-piloting the
> next 4.5.1 RC since he could be busy and I want to help with lifting some
> weights. I’m planning to do basic tests with VMware today, publish new
> systemvmtemplates tomorrow (I’m already done with KVM and Xen) and plan to
> cut a 4.5.1 RC on Monday and if we don’t hit any blocker or regressions ACS
> 4.5.1 should be released by the end of next week.
> >>
> >>> On 29-Apr-2015, at 2:49 pm, Wilder Rodrigues <
> WRodrigues@schubergphilis.com <javascript:;>> wrote:
> >>>
> >>> Hi Rohit,
> >>>
> >>> I will join you in testing 4.5 form next week - just have to finish
> some stuff.
> >>>
> >>> My environments will be:
> >>>
> >>> * Xen 6.2/6.5
> >>> * KVM (qemu 1.5.3 and 2.1)
> >>>
> >>> Let’s rock!
> >>>
> >>> Cheers,
> >>> Wilder
> >>>
> >>> On 28 Apr 2015, at 18:33, Rohit Yadav <rohit.yadav@shapeblue.com
> <javascript:;>> wrote:
> >>>
> >>>> Hi all,
> >>>>
> >>>> ### 4.5 Release Effort
> >>>>
> >>>> I’ve been exhaustively testing ACS 4.5 wrt Xen 6.2, Xen 6.5, KVM
> (qemu 2.0, 2.3) and I feel we’re pretty good but we need testing efforts
> and I want to help drive efforts to releasing ACS 4.5.1 -- if you’ve any
> issues that you would like to get fixed that are either regressions or
> blockers can you please share on this thread? If we don’t find any blockers
> or regressions let us invest in ACS 4.5 testing and release it soon. For
> other minor issues we can always fix them and release 4.5.2 etc if and when
> needed in future.
> >>>>
> >>>> Regarding my 4.5 testing - I’ve tested basic vm life cycle operations
> (deploy, start, stop, destroy/expunge, migrate to another host, migrate to
> another storage pool, deploy using iso) for all cases and found no issues;
> >>>>
> >>>> KVM with basic zone (with/without security groups, SG blockers were
> found and fixed)
> >>>> KVM with Adv zone (with/without security groups, both isolated
> network and VPC worked, networking issues were found and fixed)
> >>>> XenServer 6.2 with basic and adv zones (with/without security groups,
> no SG issues found)
> >>>> XenServer 6.5 with basic and adv zones (with/without security groups,
> SG blockers found and fixed)
> >>>>
> >>>> I found one issue that could be a blocker (if it’s not a hardware/env
> issue) -- when using security groups in advance zone with a dedicated vlan
> id (say vlan://500), I was unable to access the VMs (CPVM, SSVM, VR or user
> vms) though all normal vm_life cycles seems to work. Though this worked for
> me when I used the vlan://untagged. I tried to add vlan id 500 to my local
> nics using vconfig but I still was n’t able to do access the CPVM or user
> VMs. I guess my understanding of vlans with security groups is limited, so
> if anyone knows about this feature or has used it - please help with some
> regression testing. I plan to continue testing this week with VMWare and
> fix any issues we find.
> >>>>
> >>>>
> >>>> ### Testing against Xen, KVM, VMWare etc.
> >>>>
> >>>> With a recent improvement to allow cpu features (such as vmx) on KVM
> hosts (
> https://github.com/apache/cloudstack/commit/58cc569273905c50d089f9fd82fe80028b4e9775),
> it’s now possible to run KVM, ESX, XenServer, OVM3 (LXC, and hopefully
> others) on KVM (in case of ESX a patched qemu would be needed:
> http://people.apache.org/~bhaisaab/qemu/). Such a (ansible based) tool
> aims to be reproduce such an environment for anyone and serve as a
> developer kit (not to be confused with the DevCloud appliance, will share
> more details of the developer kit later) and as an infrastructure to run
> integration testing suite we have now. This would allow us to
> build/test/run CloudStack against various nested-virtualized hosts running
> as guests on KVM, using CloudStack.
> >>>>
> >>>> Meanwhile, Abhi is focusing on existing (integration) tests and has
> increased the number of TravisCI tests:
> https://travis-ci.org/apache/cloudstack/builds/60365947
> >>>>
> >>>> Our goal with these efforts is to (1) have a developer kit that a
> developer can have locally to build/test/develop CloudStack, (2) scale it
> up for bigger baremetal servers to run long running integration tests and
> have a better QA automation.
> >>>>
> >>>> Regards,
> >>>> Rohit Yadav
> >>>> Software Architect, ShapeBlue
> >>>> M. +91 88 262 30892 | rohit.yadav@shapeblue.com <javascript:;>
> >>>> Blog: bhaisaab.org | Twitter: @_bhaisaab
> >>>>
> >>>>
> >>>>
> >>>> Find out more about ShapeBlue and our range of CloudStack related
> services
> >>>>
> >>>> IaaS Cloud Design & Build<
> http://shapeblue.com/iaas-cloud-design-and-build//>
> >>>> CSForge – rapid IaaS deployment framework<
> http://shapeblue.com/csforge/>
> >>>> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
> >>>> CloudStack Software Engineering<
> http://shapeblue.com/cloudstack-software-engineering/>
> >>>> CloudStack Infrastructure Support<
> http://shapeblue.com/cloudstack-infrastructure-support/>
> >>>> CloudStack Bootcamp Training Courses<
> http://shapeblue.com/cloudstack-training/>
> >>>>
> >>>> This email and any attachments to it may be confidential and are
> intended solely for the use of the individual to whom it is addressed. Any
> views or opinions expressed are solely those of the author and do not
> necessarily represent those of Shape Blue Ltd or related companies. If you
> are not the intended recipient of this email, you must neither take any
> action based upon its contents, nor copy or show it to anyone. Please
> contact the sender if you believe you have received this email in error.
> Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue
> Services India LLP is a company incorporated in India and is operated under
> license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a
> company incorporated in Brasil and is operated under license from Shape
> Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of
> South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is
> a registered trademark.
> >>>
> >>
> >> Regards,
> >> Rohit Yadav
> >> Software Architect, ShapeBlue
> >> M. +91 88 262 30892 | rohit.yadav@shapeblue.com <javascript:;>
> >> Blog: bhaisaab.org | Twitter: @_bhaisaab
> >>
> >>
> >>
> >> Find out more about ShapeBlue and our range of CloudStack related
> services
> >>
> >> IaaS Cloud Design & Build<
> http://shapeblue.com/iaas-cloud-design-and-build//>
> >> CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/
> >
> >> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
> >> CloudStack Software Engineering<
> http://shapeblue.com/cloudstack-software-engineering/>
> >> CloudStack Infrastructure Support<
> http://shapeblue.com/cloudstack-infrastructure-support/>
> >> CloudStack Bootcamp Training Courses<
> http://shapeblue.com/cloudstack-training/>
> >>
> >> This email and any attachments to it may be confidential and are
> intended solely for the use of the individual to whom it is addressed. Any
> views or opinions expressed are solely those of the author and do not
> necessarily represent those of Shape Blue Ltd or related companies. If you
> are not the intended recipient of this email, you must neither take any
> action based upon its contents, nor copy or show it to anyone. Please
> contact the sender if you believe you have received this email in error.
> Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue
> Services India LLP is a company incorporated in India and is operated under
> license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a
> company incorporated in Brasil and is operated under license from Shape
> Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of
> South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is
> a registered trademark.
>
> Find out more about ShapeBlue and our range of CloudStack related services
>
> IaaS Cloud Design & Build<
> http://shapeblue.com/iaas-cloud-design-and-build//>
> CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
> CloudStack Software Engineering<
> http://shapeblue.com/cloudstack-software-engineering/>
> CloudStack Infrastructure Support<
> http://shapeblue.com/cloudstack-infrastructure-support/>
> CloudStack Bootcamp Training Courses<
> http://shapeblue.com/cloudstack-training/>
>
> This email and any attachments to it may be confidential and are intended
> solely for the use of the individual to whom it is addressed. Any views or
> opinions expressed are solely those of the author and do not necessarily
> represent those of Shape Blue Ltd or related companies. If you are not the
> intended recipient of this email, you must neither take any action based
> upon its contents, nor copy or show it to anyone. Please contact the sender
> if you believe you have received this email in error. Shape Blue Ltd is a
> company incorporated in England & Wales. ShapeBlue Services India LLP is a
> company incorporated in India and is operated under license from Shape Blue
> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil
> and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is
> a company registered by The Republic of South Africa and is traded under
> license from Shape Blue Ltd. ShapeBlue is a registered trademark.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message