cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Nalley <da...@gnsa.us>
Subject Re: [DISCUSS] State of 4.5 and testing in local environments
Date Thu, 30 Apr 2015 02:12:33 GMT
The problem is really two fold.
Cutting an RC isn't terribly time consuming, once you've done it a
dozen times or so, it goes by pretty quickly.

The problem is the time it takes to do any material testing; and that
we don't have a way of telling what the status of a candidate is
without doing lots of manual testing. We've had multiple cases of
having enough binding votes for 4.5.x candidates only for someone to
find problems. This leads me (and this could just be my perspective)
to let an RC sit for a while - get folks to test it. My experience as
a person voting is that rapid churn of RCs leads me to not even
attempt to setup a testing environment, much less do any testing. I
found that by the time I could get around to testing an RC vote had
been cancelled and it had been rerolled, sometimes twice.

I've said this before, and I'll toss it out again. I think we have to
start 'trusting' the automated tests. We find lots of problems
manually, but we don't improve the testing situation for those blocker
issues that stopped us from releasing. I'm of the opinion we should
release quickly, very quickly. But if we have to spend a week just
looking for bugs, that will never fly. So - let the standard be
automated tests - and if there's a problem, we create an automated
test for it, and solve in a week or two. That will force us to live
and die by automated tests.

--David

On Wed, Apr 29, 2015 at 9:45 AM, Rohit Yadav <rohit.yadav@shapeblue.com> wrote:
> Hi Wilder - Thanks we’ll need all the ammo we have :) I’ve already done my testings
wrt XS 6.2/6.5 and KVM qemu 2.0/2.1/2.2 so will test KVM 1.5 and VMware 5.5.
>
> Hi Daan - makes sense, we’re “so" late with 4.5.1 that we need to focus on 4.6.0
to avoid delaying it. I’ve asked David regarding co-piloting the next 4.5.1 RC since he
could be busy and I want to help with lifting some weights. I’m planning to do basic tests
with VMware today, publish new systemvmtemplates tomorrow (I’m already done with KVM and
Xen) and plan to cut a 4.5.1 RC on Monday and if we don’t hit any blocker or regressions
ACS 4.5.1 should be released by the end of next week.
>
>> On 29-Apr-2015, at 2:49 pm, Wilder Rodrigues <WRodrigues@schubergphilis.com>
wrote:
>>
>> Hi Rohit,
>>
>> I will join you in testing 4.5 form next week - just have to finish some stuff.
>>
>> My environments will be:
>>
>> * Xen 6.2/6.5
>> * KVM (qemu 1.5.3 and 2.1)
>>
>> Let’s rock!
>>
>> Cheers,
>> Wilder
>>
>> On 28 Apr 2015, at 18:33, Rohit Yadav <rohit.yadav@shapeblue.com> wrote:
>>
>>> Hi all,
>>>
>>> ### 4.5 Release Effort
>>>
>>> I’ve been exhaustively testing ACS 4.5 wrt Xen 6.2, Xen 6.5, KVM (qemu 2.0,
2.3) and I feel we’re pretty good but we need testing efforts and I want to help drive efforts
to releasing ACS 4.5.1 -- if you’ve any issues that you would like to get fixed that are
either regressions or blockers can you please share on this thread? If we don’t find any
blockers or regressions let us invest in ACS 4.5 testing and release it soon. For other minor
issues we can always fix them and release 4.5.2 etc if and when needed in future.
>>>
>>> Regarding my 4.5 testing - I’ve tested basic vm life cycle operations (deploy,
start, stop, destroy/expunge, migrate to another host, migrate to another storage pool, deploy
using iso) for all cases and found no issues;
>>>
>>> KVM with basic zone (with/without security groups, SG blockers were found and
fixed)
>>> KVM with Adv zone (with/without security groups, both isolated network and VPC
worked, networking issues were found and fixed)
>>> XenServer 6.2 with basic and adv zones (with/without security groups, no SG issues
found)
>>> XenServer 6.5 with basic and adv zones (with/without security groups, SG blockers
found and fixed)
>>>
>>> I found one issue that could be a blocker (if it’s not a hardware/env issue)
-- when using security groups in advance zone with a dedicated vlan id (say vlan://500), I
was unable to access the VMs (CPVM, SSVM, VR or user vms) though all normal vm_life cycles
seems to work. Though this worked for me when I used the vlan://untagged. I tried to add vlan
id 500 to my local nics using vconfig but I still was n’t able to do access the CPVM or
user VMs. I guess my understanding of vlans with security groups is limited, so if anyone
knows about this feature or has used it - please help with some regression testing. I plan
to continue testing this week with VMWare and fix any issues we find.
>>>
>>>
>>> ### Testing against Xen, KVM, VMWare etc.
>>>
>>> With a recent improvement to allow cpu features (such as vmx) on KVM hosts (https://github.com/apache/cloudstack/commit/58cc569273905c50d089f9fd82fe80028b4e9775),
it’s now possible to run KVM, ESX, XenServer, OVM3 (LXC, and hopefully others) on KVM (in
case of ESX a patched qemu would be needed: http://people.apache.org/~bhaisaab/qemu/). Such
a (ansible based) tool aims to be reproduce such an environment for anyone and serve as a
developer kit (not to be confused with the DevCloud appliance, will share more details of
the developer kit later) and as an infrastructure to run integration testing suite we have
now. This would allow us to build/test/run CloudStack against various nested-virtualized hosts
running as guests on KVM, using CloudStack.
>>>
>>> Meanwhile, Abhi is focusing on existing (integration) tests and has increased
the number of TravisCI tests: https://travis-ci.org/apache/cloudstack/builds/60365947
>>>
>>> Our goal with these efforts is to (1) have a developer kit that a developer can
have locally to build/test/develop CloudStack, (2) scale it up for bigger baremetal servers
to run long running integration tests and have a better QA automation.
>>>
>>> Regards,
>>> Rohit Yadav
>>> Software Architect, ShapeBlue
>>> M. +91 88 262 30892 | rohit.yadav@shapeblue.com
>>> Blog: bhaisaab.org | Twitter: @_bhaisaab
>>>
>>>
>>>
>>> Find out more about ShapeBlue and our range of CloudStack related services
>>>
>>> IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//>
>>> CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
>>> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
>>> CloudStack Software Engineering<http://shapeblue.com/cloudstack-software-engineering/>
>>> CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/>
>>> CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
>>>
>>> This email and any attachments to it may be confidential and are intended solely
for the use of the individual to whom it is addressed. Any views or opinions expressed are
solely those of the author and do not necessarily represent those of Shape Blue Ltd or related
companies. If you are not the intended recipient of this email, you must neither take any
action based upon its contents, nor copy or show it to anyone. Please contact the sender if
you believe you have received this email in error. Shape Blue Ltd is a company incorporated
in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and
is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company
incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty
Ltd is a company registered by The Republic of South Africa and is traded under license from
Shape Blue Ltd. ShapeBlue is a registered trademark.
>>
>
> Regards,
> Rohit Yadav
> Software Architect, ShapeBlue
> M. +91 88 262 30892 | rohit.yadav@shapeblue.com
> Blog: bhaisaab.org | Twitter: @_bhaisaab
>
>
>
> Find out more about ShapeBlue and our range of CloudStack related services
>
> IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//>
> CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
> CloudStack Software Engineering<http://shapeblue.com/cloudstack-software-engineering/>
> CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/>
> CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
>
> This email and any attachments to it may be confidential and are intended solely for
the use of the individual to whom it is addressed. Any views or opinions expressed are solely
those of the author and do not necessarily represent those of Shape Blue Ltd or related companies.
If you are not the intended recipient of this email, you must neither take any action based
upon its contents, nor copy or show it to anyone. Please contact the sender if you believe
you have received this email in error. Shape Blue Ltd is a company incorporated in England
& Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated
under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated
in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company
registered by The Republic of South Africa and is traded under license from Shape Blue Ltd.
ShapeBlue is a registered trademark.

Mime
View raw message