cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Angus <paul.an...@shapeblue.com>
Subject RE: [Discuss] Flexible CI/CD - Trillian
Date Wed, 03 Feb 2016 09:35:43 GMT
Hi Nux,

Right now, yes. Rohit has done a lot of work to nest hypervisors under KVM, but has found
shortfalls in it capabilities which has meant we can't do 'everything' ( I don't know the
details).




Paul Angus
VP Technology   ,       ShapeBlue


t:      @cloudyangus<tel:@cloudyangus>

e:      paul.angus@shapeblue.com<mailto:paul.angus@shapeblue.com>        |      w: 
    www.shapeblue.com<http://www.shapeblue.com>





-----Original Message-----
From: Nux! [mailto:nux@li.nux.ro]
Sent: 02 February 2016 11:14
To: dev@cloudstack.apache.org
Cc: users@cloudstack.apache.org
Subject: Re: [Discuss] Flexible CI/CD - Trillian

Hi Paul,

This is great news, good job, naming apart. :-)

Is ESXi a hard requirement on running Trillian?

--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro

----- Original Message -----
> From: "Paul Angus" <paul.angus@shapeblue.com>
> To: dev@cloudstack.apache.org
> Cc: users@cloudstack.apache.org
> Sent: Tuesday, 2 February, 2016 10:49:46
> Subject: [Discuss] Flexible CI/CD - Trillian

> Project: Trillian
> We have been working on PoC of a CI environment design which will:
>
> · Provide fast build or rebuild of environments for testing.
>
> · Enable multiple independent concurrent builds
>
> · Be available on-demand through automation or individual request.
>
> · Be capable of fully utilising all available hardware
>
> · Flexible enough to be used to build super-realistic development
> environments.
> We intend to contribute and maintain our work within the Apache repos.
> However, we are currently building the POC, figuring out the
> requirements (and quirks) of the individual pieces, before pushing
> something concrete for to the community to review.
>
> We envision that Trillian would cater for a number of use cases:
>
> 1. CloudStack community integration testing of master against multiple
> deployment scenarios (using ASF infra)
>
> 2. CloudStack community integration testing of PRs against multiple
> deployment scenarios (using ASF infra)
>
> 3. Organisations/individuals running the full suites of tests available in
> Marvin against any physical environment they have.
>
> 4. Organisations/individuals deploying and running the full suites of
> tests available in Marvin against virtualised infrastructures which
> can be deployed by Marvin.
> As we intend Trillian to test multiple environments concurrently, we
> use nested virtualization on ESXi hosts (our testing has shown that
> this is the only hypervisor which can support the nested
> virtualisation of all other hypervisors with reasonable performance).
> We use Ansible to deploy and configure all aspects of the build as
> this will greatly lower the barrier to entry for independent testers.
> We use CloudStack to provision the management server and virtualised
> (nested) hosts on the physical hosts. We are creating Ansible
> playbooks and roles which
> can:
>
> 1. Create guest instances using Rene’s Ansible 2.0 CloudStack modules - a
> Marvin VM, a Mgmt Server (CentOS or Ubuntu), any number of compute
> hosts (KVM, vSphere or XenServer. Hyper-V later)
>
> 2. Configure hosts (inc. installing the relevant CloudStack agent where
> required)
>
> 3. Install required ACS packages on management server
>
> 4. Configure a zone (including adding the compute hosts) via Marvin.
>
> 5. Run the required Marvin tests.
>
> 6. Return the results
> We may need to propose enhancements to Marvin in order to sync the
> configuration of hosts with the configuration used by Marvin.
>
> Using virtualised test environments, we can have multiple test
> scenarios running concurrently. To do this we have found that it is
> necessary to create pools or ranges of VLANs and IP addresses and
> allocate them to environments. So for any given physical environment
> which will be used for testing in, we take the total
> range(s) of IPs and VLANs available and carve them into
> non-overlapping chunks suitable for concurrent use as mgmt, public and
> guest networks. These are stored in a MariaDB database. When a range
> is being used in a testing environment, that range is marked as
> ‘inuse’ in the database. When creating a test environment, Trillian
> looks in the database for the next available VLAN range, the next
> available public IP range and so on. The returned values are used to
> populate a Marvin cfg file which in turn will be used to both build
> the environment and when running the Marvin testing. When the
> virtualised infra is cleaned up, the database will be updated to reflect that the used
ranges are available again.
> This initiative has only recently been started, and as stated earlier
> we are currently figuring out the requirements (and quirks) of the
> individual pieces and looking for the most suitable wrapper to glue it all together.
> Also I have found that Marvin requires a little work to make the
> output more meaningful/readable (especially in the case of errors and
> exceptions) and to make it a little more intelligent about the tests
> it can/can’t run based on the chosen infrastructure components. I have
> also found unreachable or very slow ISO and template paths hardcoded into Marvin or individual
tests.
> We plan to enhance tests to address these issues and also reduce
> runtimes where possible.
>
> NEXT STEPS
>
> We will continue to our work on the PoC and share the results.
>
> We’d like to get some community agreement around the goals and use
> cases which we are planning to satisfy before we get too far down the
> road of our PoC. So please share your thoughts.
>
>
>
> [ShapeBlue]<http://www.shapeblue.com>
> Paul Angus
> VP Technology , ShapeBlue
>
>
> d: +44 203 617 0528 | s: +44 203 603
> 0540<tel:+44%20203%20617%200528%20|%20s:%20+44%20203%20603%200540> |
> m: +44 7711 418784<tel:+44%207711%20418784>
>
> e: paul.angus@shapeblue.com | t:
> @cloudyangus<mailto:paul.angus@shapeblue.com%20|%20t:%20@cloudyangus> |
> w: www.shapeblue.com<http://www.shapeblue.com>
>
> a: 53 Chandos Place, Covent Garden London WC2N 4HS UK
>
>
> [cid:imagea932fa.png@ce3f6520.47a350c4]
>
>
> Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue
> Services India LLP is a company incorporated in India and is operated
> under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda
> is a company incorporated in Brasil and is operated under license from
> Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The
> Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is
a registered trademark.
> This email and any attachments to it may be confidential and are
> intended solely for the use of the individual to whom it is addressed.
> Any views or opinions expressed are solely those of the author and do
> not necessarily represent those of Shape Blue Ltd or related
> companies. If you are not the intended recipient of this email, you
> must neither take any action based upon its contents, nor copy or show
> it to anyone. Please contact the sender if you believe you have received this email in
error.
>
>
>
>
> Find out more about ShapeBlue and our range of CloudStack related services:
> IaaS Cloud Design &
> Build<http://shapeblue.com/iaas-cloud-design-and-build//> | CSForge –
> rapid IaaS deployment framework<http://shapeblue.com/csforge/>
> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/> |
> CloudStack Software
> Engineering<http://shapeblue.com/cloudstack-software-engineering/>
> CloudStack Infrastructure
> Support<http://shapeblue.com/cloudstack-infrastructure-support/> |
> CloudStack Bootcamp Training
> Courses<http://shapeblue.com/cloudstack-training/>
Find out more about ShapeBlue and our range of CloudStack related services:
IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//> |
CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/> | CloudStack Software
Engineering<http://shapeblue.com/cloudstack-software-engineering/>
CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/>
| CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message