cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Remi Bergsma <r...@remi.nl>
Subject Re: [DISCUSS] State of 4.5 and testing in local environments
Date Thu, 30 Apr 2015 04:59:43 GMT
+1 Totally agree and willing to help make it happen!

Sent from my iPhone

> On 30 Apr 2015, at 04:12, David Nalley <david@gnsa.us> wrote:
> 
> The problem is really two fold.
> Cutting an RC isn't terribly time consuming, once you've done it a
> dozen times or so, it goes by pretty quickly.
> 
> The problem is the time it takes to do any material testing; and that
> we don't have a way of telling what the status of a candidate is
> without doing lots of manual testing. We've had multiple cases of
> having enough binding votes for 4.5.x candidates only for someone to
> find problems. This leads me (and this could just be my perspective)
> to let an RC sit for a while - get folks to test it. My experience as
> a person voting is that rapid churn of RCs leads me to not even
> attempt to setup a testing environment, much less do any testing. I
> found that by the time I could get around to testing an RC vote had
> been cancelled and it had been rerolled, sometimes twice.
> 
> I've said this before, and I'll toss it out again. I think we have to
> start 'trusting' the automated tests. We find lots of problems
> manually, but we don't improve the testing situation for those blocker
> issues that stopped us from releasing. I'm of the opinion we should
> release quickly, very quickly. But if we have to spend a week just
> looking for bugs, that will never fly. So - let the standard be
> automated tests - and if there's a problem, we create an automated
> test for it, and solve in a week or two. That will force us to live
> and die by automated tests.
> 
> --David
> 
>> On Wed, Apr 29, 2015 at 9:45 AM, Rohit Yadav <rohit.yadav@shapeblue.com> wrote:
>> Hi Wilder - Thanks we’ll need all the ammo we have :) I’ve already done my testings
wrt XS 6.2/6.5 and KVM qemu 2.0/2.1/2.2 so will test KVM 1.5 and VMware 5.5.
>> 
>> Hi Daan - makes sense, we’re “so" late with 4.5.1 that we need to focus on 4.6.0
to avoid delaying it. I’ve asked David regarding co-piloting the next 4.5.1 RC since he
could be busy and I want to help with lifting some weights. I’m planning to do basic tests
with VMware today, publish new systemvmtemplates tomorrow (I’m already done with KVM and
Xen) and plan to cut a 4.5.1 RC on Monday and if we don’t hit any blocker or regressions
ACS 4.5.1 should be released by the end of next week.
>> 
>>> On 29-Apr-2015, at 2:49 pm, Wilder Rodrigues <WRodrigues@schubergphilis.com>
wrote:
>>> 
>>> Hi Rohit,
>>> 
>>> I will join you in testing 4.5 form next week - just have to finish some stuff.
>>> 
>>> My environments will be:
>>> 
>>> * Xen 6.2/6.5
>>> * KVM (qemu 1.5.3 and 2.1)
>>> 
>>> Let’s rock!
>>> 
>>> Cheers,
>>> Wilder
>>> 
>>>> On 28 Apr 2015, at 18:33, Rohit Yadav <rohit.yadav@shapeblue.com> wrote:
>>>> 
>>>> Hi all,
>>>> 
>>>> ### 4.5 Release Effort
>>>> 
>>>> I’ve been exhaustively testing ACS 4.5 wrt Xen 6.2, Xen 6.5, KVM (qemu
2.0, 2.3) and I feel we’re pretty good but we need testing efforts and I want to help drive
efforts to releasing ACS 4.5.1 -- if you’ve any issues that you would like to get fixed
that are either regressions or blockers can you please share on this thread? If we don’t
find any blockers or regressions let us invest in ACS 4.5 testing and release it soon. For
other minor issues we can always fix them and release 4.5.2 etc if and when needed in future.
>>>> 
>>>> Regarding my 4.5 testing - I’ve tested basic vm life cycle operations (deploy,
start, stop, destroy/expunge, migrate to another host, migrate to another storage pool, deploy
using iso) for all cases and found no issues;
>>>> 
>>>> KVM with basic zone (with/without security groups, SG blockers were found
and fixed)
>>>> KVM with Adv zone (with/without security groups, both isolated network and
VPC worked, networking issues were found and fixed)
>>>> XenServer 6.2 with basic and adv zones (with/without security groups, no
SG issues found)
>>>> XenServer 6.5 with basic and adv zones (with/without security groups, SG
blockers found and fixed)
>>>> 
>>>> I found one issue that could be a blocker (if it’s not a hardware/env issue)
-- when using security groups in advance zone with a dedicated vlan id (say vlan://500), I
was unable to access the VMs (CPVM, SSVM, VR or user vms) though all normal vm_life cycles
seems to work. Though this worked for me when I used the vlan://untagged. I tried to add vlan
id 500 to my local nics using vconfig but I still was n’t able to do access the CPVM or
user VMs. I guess my understanding of vlans with security groups is limited, so if anyone
knows about this feature or has used it - please help with some regression testing. I plan
to continue testing this week with VMWare and fix any issues we find.
>>>> 
>>>> 
>>>> ### Testing against Xen, KVM, VMWare etc.
>>>> 
>>>> With a recent improvement to allow cpu features (such as vmx) on KVM hosts
(https://github.com/apache/cloudstack/commit/58cc569273905c50d089f9fd82fe80028b4e9775), it’s
now possible to run KVM, ESX, XenServer, OVM3 (LXC, and hopefully others) on KVM (in case
of ESX a patched qemu would be needed: http://people.apache.org/~bhaisaab/qemu/). Such a (ansible
based) tool aims to be reproduce such an environment for anyone and serve as a developer kit
(not to be confused with the DevCloud appliance, will share more details of the developer
kit later) and as an infrastructure to run integration testing suite we have now. This would
allow us to build/test/run CloudStack against various nested-virtualized hosts running as
guests on KVM, using CloudStack.
>>>> 
>>>> Meanwhile, Abhi is focusing on existing (integration) tests and has increased
the number of TravisCI tests: https://travis-ci.org/apache/cloudstack/builds/60365947
>>>> 
>>>> Our goal with these efforts is to (1) have a developer kit that a developer
can have locally to build/test/develop CloudStack, (2) scale it up for bigger baremetal servers
to run long running integration tests and have a better QA automation.
>>>> 
>>>> Regards,
>>>> Rohit Yadav
>>>> Software Architect, ShapeBlue
>>>> M. +91 88 262 30892 | rohit.yadav@shapeblue.com
>>>> Blog: bhaisaab.org | Twitter: @_bhaisaab
>>>> 
>>>> 
>>>> 
>>>> Find out more about ShapeBlue and our range of CloudStack related services
>>>> 
>>>> IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//>
>>>> CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
>>>> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
>>>> CloudStack Software Engineering<http://shapeblue.com/cloudstack-software-engineering/>
>>>> CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/>
>>>> CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
>>>> 
>>>> This email and any attachments to it may be confidential and are intended
solely for the use of the individual to whom it is addressed. Any views or opinions expressed
are solely those of the author and do not necessarily represent those of Shape Blue Ltd or
related companies. If you are not the intended recipient of this email, you must neither take
any action based upon its contents, nor copy or show it to anyone. Please contact the sender
if you believe you have received this email in error. Shape Blue Ltd is a company incorporated
in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and
is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company
incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty
Ltd is a company registered by The Republic of South Africa and is traded under license from
Shape Blue Ltd. ShapeBlue is a registered trademark.
>> 
>> Regards,
>> Rohit Yadav
>> Software Architect, ShapeBlue
>> M. +91 88 262 30892 | rohit.yadav@shapeblue.com
>> Blog: bhaisaab.org | Twitter: @_bhaisaab
>> 
>> 
>> 
>> Find out more about ShapeBlue and our range of CloudStack related services
>> 
>> IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//>
>> CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
>> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
>> CloudStack Software Engineering<http://shapeblue.com/cloudstack-software-engineering/>
>> CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/>
>> CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
>> 
>> This email and any attachments to it may be confidential and are intended solely
for the use of the individual to whom it is addressed. Any views or opinions expressed are
solely those of the author and do not necessarily represent those of Shape Blue Ltd or related
companies. If you are not the intended recipient of this email, you must neither take any
action based upon its contents, nor copy or show it to anyone. Please contact the sender if
you believe you have received this email in error. Shape Blue Ltd is a company incorporated
in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and
is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company
incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty
Ltd is a company registered by The Republic of South Africa and is traded under license from
Shape Blue Ltd. ShapeBlue is a registered trademark.

Mime
View raw message