cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Will Stevens <wstev...@cloudops.com>
Subject Re: Important Pending Items
Date Tue, 01 Mar 2016 20:50:32 GMT
Thanks Paul.  Great to hear you are planning to make everything available
and you are willing to take contributions from the community.

I am currently trying to get up to speed on everything that has been done
and all the different pieces in play.  I would like to have a CI in place
(even if it is only locally on my hardware) for testing PRs for 4.9.  My
focus right now is solving the CI problem, so I am interested in any
developments on this front.

We have Travis doing smoke tests on every PR right now.  Travis is free for
open source projects.  Is there a reason we are not doing more extensive
tests with Travis since it is already integrated into github?  Like maybe
running the simulator on every PR?  I think this could be a good first step
to weed out some of the PRs that are likely to fail a full CI run.

Cheers,

*Will STEVENS*
Lead Developer

*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_

On Tue, Mar 1, 2016 at 3:40 PM, Paul Angus <paul.angus@shapeblue.com> wrote:

> Hi Will,
>
> It will be 100% open source. The $dayjob has blocked the team from working
> on it recently. I'd like to get some of our ideas into the CloudStack wiki
> this week before I go on leave for a week.
> Rohit has created a repo for us: https://github.com/shapeblue/Trillian
> (which looks a bit sad at the moment - I hope to push our Ansible roles
> for building mgmt. servers and Marvin boxes into it this week also)
>
> Internally we're planning a two week sprint on it starting 14th March.
> Having to dip in and out of this effort is making it very difficult to get
> anywhere
>
> I think the biggest part of the work will be to get Marvin up to scratch,
> the log output from the tests is pretty bad.
>
> We very much want this to be a community 'thing', and we're happy to work
> with everyone/anyone to get there.
>
> Any ideas on the most effective way to have a number of people on this -
> I'm all ears.
> In the meantime I'll try to make good on my promises to get our ideas
> visible to everyone...
>
>
>
>
> Paul Angus
> VP Technology   ,       ShapeBlue
>
>
> t:      @cloudyangus<tel:@cloudyangus>
>
> e:      paul.angus@shapeblue.com<mailto:paul.angus@shapeblue.com>
> |      w:      www.shapeblue.com<http://www.shapeblue.com>
>
>
>
>
>
> -----Original Message-----
> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
> Behalf Of Will Stevens
> Sent: Monday, February 29, 2016 9:52 PM
> To: dev@cloudstack.apache.org
> Subject: Re: Important Pending Items
>
> @Paul and @Bharat, thank you for the quality write-ups on this topic.
>
> re:Paul's Details:
> I share a lot of your thoughts on this. The hypervisor in hypervisor setup
> with the ability to test multiple environments is also core to my thinking.
> I can't find any details on Trillian, other than what is in this thread. Is
> it open source and freely available? This is being developed at ShapeBlue?
>
> re:Bharat's Details:
> This seems like it is the most likely candidate for being able to offer
> value in the short term. Thank you for the work documenting it and making
> the source available. I still need to review the code. This approach will
> take quite a bit of work get setup initially, but once setup, it looks like
> it will do the trick. By using only freely available software makes it more
> accessible than some options which have been suggested in the past.
>
> My thoughts on the topic:
> - I think it is important that the CI can test all the different
> hypervisors and any hardware integrations available in the environment.
> - Ideally the CI tool would be easily distributed and installed at
> multiple sites. Every company who depends on ACS will have their own
> configuration, hardware and mission critical features. If the CI can be
> distributed (as independent installs), it will reduce the load on any
> specific site and it will enable companies who have specific hardware in
> their direct path to success the ability to validate their use case.
> - I think a standardized CI output is also important. Ideally the summary
> of the CI run would be posted back to the PR thread. I think it is also
> important to push the detailed logs to a central location so others can
> review them. Ideally, links to the detailed logs would also be pushed to
> the PR thread.
> - I think there should be multiple tiers of testing. Basic simulator
> testing and full runs. We may be able to setup simulator testing using a
> Docker container. I think @pdion891 has put some work into this that I may
> be able to build on.
> - I think it is probably too ambitious to do CI on every PR before merge,
> but ideally we would We may have to only CI the master branch for now and
> revert any PR that causes it to fail.
>
> I have been digging into this quite a bit today. I will continue working
> on this as it is very important to me for the 4.9 release. I would like to
> setup a CI environment locally as well to help validate and contribute to
> the effort. I have some hardware I can throw at this, so I will be working
> to get something setup to start testing ASAP.
>
> If I can work off either (or both) of your work, I would appreciate it.
>
> Cheers,
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> @CloudOps_
>
> On Fri, Feb 12, 2016 at 12:49 AM, Bharat Kumar <bharat.kumar@citrix.com>
> wrote:
>
> > Hi Paul,
> >
> > The goals and functionality of the CI on which we are working are the
> > same (please review the FS<
> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Test+bed+orches
> > trator+and+test+runner+to+enable+continuos+integration>
> > ), The difference being the method of implementation. We are not using
> > nested virtualisation to create test environments. The work on CI
> > began long back but unfortunately due to other priorities we could not
> > work on a continuous basis and bring it to completion.
> >
> > I agree that we should come to a consensus on what the community wants
> > from the CI. IMHO the basic and immediate requirement would be to have
> > a stable system to test the PRs and post results. Once we have this,
> > we can talk about providing test setups to the community and next
> > steps. I think the availability of hardware has been a big hurdle
> > (even if we use virtualisation).
> >
> > Instead of working independently on the same thing let us work on this
> > together. I will share what we have currently, we can use it, or parts
> > of it (some of it is there at
> > https://github.com/bvbharatk/cloudstack-automation, but it is not
> > organised as in cannot be run out of the box). While virtualisation is
> > good from the ease of orchestration and resource utilisation point of
> > view, it has other problems like compatibility/dependencies issues.
> > Also it might tie the CI to a particular Hypervisor like ESXi as you
> have pointed out.
> >
> > So let us discuss on how to proceed further, and get things rolling.
> >
> > Thanks,
> > Bharat.
> >
> >
> > On 11-Feb-2016, at 9:44 PM, Paul Angus <paul.angus@shapeblue.com<mailto:
> > paul.angus@shapeblue.com>> wrote:
> >
> > Hi Bharat,
> >
> > Early last week I opened a discussion thread around the aims of
> > Trillian which we had begun working on as a CI/CD platform for the
> > community hardware (as well as individual use). Please could you
> > elaborate on where there is/isn't cross-over? In terms of
> > functionality and goals rather than method. It would be great to get
> > community consensus around what we want from our centralised CI/CD.
> >
> >
> > Project: Trillian
> > We have been working on PoC of a CI environment design which will:
> > * Provide fast build or rebuild of environments for testing.
> > * Enable multiple independent concurrent builds
> > * Be available on-demand through automation or individual request.
> > * Be capable of fully utilising all available hardware
> > * Flexible enough to be used to build super-realistic development
> > environments.
> > We intend to contribute and maintain our work within the Apache repos.
> > However, we are currently building the POC, figuring out the
> > requirements (and quirks) of the individual pieces, before pushing
> > something concrete for to the community to review.
> >
> > We envision that Trillian would cater for a number of use cases:
> > 1. CloudStack community integration testing of master against multiple
> > deployment scenarios (using ASF infra) 2. CloudStack community
> > integration testing of PRs against multiple deployment scenarios
> > (using ASF infra) 3. Organisations/individuals running the full suites
> > of tests available in Marvin against any physical environment they
> > have.
> > 4. Organisations/individuals deploying and running the full suites of
> > tests available in Marvin against virtualised infrastructures which
> > can be deployed by Marvin.
> > As we intend Trillian to test multiple environments concurrently, we
> > use nested virtualization on ESXi hosts (our testing has shown that
> > this is the only hypervisor which can support the nested
> > virtualisation of all other hypervisors with reasonable performance).
> > We use Ansible to deploy and configure all aspects of the build as
> > this will greatly lower the barrier to entry for independent testers.
> > We use CloudStack to provision the management server and virtualised
> > (nested) hosts on the physical hosts. We are creating Ansible
> > playbooks and roles which can:
> > 1. Create guest instances using Rene's Ansible 2.0 CloudStack modules
> > - a Marvin VM, a Mgmt Server (CentOS or Ubuntu), any number of compute
> > hosts (KVM, vSphere or XenServer. Hyper-V later) 2. Configure hosts
> > (inc. installing the relevant CloudStack agent where
> > required)
> > 3. Install required ACS packages on management server 4. Configure a
> > zone (including adding the compute hosts) via Marvin.
> > 5. Run the required Marvin tests.
> > 6. Return the results
> > We may need to propose enhancements to Marvin in order to sync the
> > configuration of hosts with the configuration used by Marvin.
> >
> > Using virtualised test environments, we can have multiple test
> > scenarios running concurrently. To do this we have found that it is
> > necessary to create pools or ranges of VLANs and IP addresses and
> > allocate them to environments. So for any given physical environment
> > which will be used for testing in, we take the total range(s) of IPs
> > and VLANs available and carve them into non-overlapping chunks
> > suitable for concurrent use as mgmt, public and guest networks. These
> > are stored in a MariaDB database. When a range is being used in a
> > testing environment, that range is marked as 'inuse' in the database.
> > When creating a test environment, Trillian looks in the database for
> > the next available VLAN range, the next available public IP range and
> > so on. The returned values are used to populate a Marvin cfg file
> > which in turn will be used to both build the environment and when
> > running the Marvin testing. When the virtualised infra is cleaned up,
> > the database will be updated to reflect that the used ranges are
> available again.
> > This initiative has only recently been started, and as stated earlier
> > we are currently figuring out the requirements (and quirks) of the
> > individual pieces and looking for the most suitable wrapper to glue it
> all together.
> > Also I have found that Marvin requires a little work to make the
> > output more meaningful/readable (especially in the case of errors and
> > exceptions) and to make it a little more intelligent about the tests
> > it can/can't run based on the chosen infrastructure components. I have
> > also found unreachable or very slow ISO and template paths hardcoded
> > into Marvin or individual tests.
> > We plan to enhance tests to address these issues and also reduce
> > runtimes where possible.
> >
> >
> > <http://www.shapeblue.com/>
> > Paul Angus
> > VP Technology , ShapeBlue
> >
> >
> > d: +44 203 617 0528 | s: +44 203 603 0540
> <tel:+44%20203%20617%200528%20|%20s:%20+44%20203%20603%200540>
> > | m: +44 7711 418784<tel:+44%207711%20418784>
> >
> > e: paul.angus@shapeblue.com | t: @cloudyangus<mailto:
> > paul.angus@shapeblue.com%20|%20t:%20@cloudyangus> | w:
> > www.shapeblue.com<http://www.shapeblue.com/>
> >
> > a: 53 Chandos Place, Covent Garden London WC2N 4HS UK
> >
> >
> >
> >
> >
> > Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue
> > Services India LLP is a company incorporated in India and is operated
> > under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda
> > is a company incorporated in Brasil and is operated under license from
> > Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The
> > Republic of South Africa and is traded under license from Shape Blue
> > Ltd. ShapeBlue is a registered trademark.
> > This email and any attachments to it may be confidential and are
> > intended solely for the use of the individual to whom it is addressed.
> > Any views or opinions expressed are solely those of the author and do
> > not necessarily represent those of Shape Blue Ltd or related
> > companies. If you are not the intended recipient of this email, you
> > must neither take any action based upon its contents, nor copy or show
> > it to anyone. Please contact the sender if you believe you have received
> this email in error.
> >
> >
> >
> >
> > -----Original Message-----
> > From: Bharat Kumar [mailto:bharat.kumar@citrix.com]
> > Sent: Thursday, February 11, 2016 9:57 AM
> > To: dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>
> > Subject: Re: Important Pending Items
> >
> > Hi Sebastien,
> >
> > As Raja said, we are actively working on it and we will share the code
> > on github soon.
> >
> > Thanks,
> > Bharat.
> >
> >
> > > On 11-Feb-2016, at 2:41 PM, Raja Pullela <raja.pullela@citrix.com
> > <mailto:raja.pullela@citrix.com>> wrote:
> > >
> > > Hi Sebastien,
> > >
> > > On item (3) - the BVTs have been running consistently (with same
> > passrates) and haven't had the time to automate the process of
> > reporting to an external site.
> > > Bharat and Sanjeev are working on CI - running BVTs/Regression,
> > > working
> > on getting it run automatically and consistently.
> > > Hope to see it ready soon.
> > >
> > > Best,
> > > Raja
> > > -----Original Message-----
> > > From: Sebastien Goasguen [mailto:runseb@gmail.com]
> > > Sent: Wednesday, February 10, 2016 2:29 PM
> > > To: dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>
> > > Subject: Important Pending Items
> > >
> > > Morning folks,
> > >
> > > We have several crucial pending items, that we need to resolve
> > > before
> > moving on:
> > >
> > > 1- We need an RM for master ( just saw some commits there that
> > > should be
> > reverted or merged properly in other branches).
> > >
> > > 2- We need to automate writing release notes, pushing/tagging new
> > > docs
> > when release come out and announcing releases on website. Currently
> > neither
> > 4.7 nor 4.8 have been announced.
> > >
> > > 3- CI is still almost inexistent
> > >
> > > -Sebastien
> >
> > Find out more about ShapeBlue and our range of CloudStack related
> services:
> > IaaS Cloud Design & Build<
> > http://shapeblue.com/iaas-cloud-design-and-build//> | CSForge – rapid
> > IaaS deployment framework<http://shapeblue.com/csforge/>
> > CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/> |
> > CloudStack Software Engineering<
> > http://shapeblue.com/cloudstack-software-engineering/>
> > CloudStack Infrastructure Support<
> > http://shapeblue.com/cloudstack-infrastructure-support/> | CloudStack
> > Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
> >
> >
> Find out more about ShapeBlue and our range of CloudStack related services:
> IaaS Cloud Design & Build<
> http://shapeblue.com/iaas-cloud-design-and-build//> | CSForge – rapid
> IaaS deployment framework<http://shapeblue.com/csforge/>
> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/> |
> CloudStack Software Engineering<
> http://shapeblue.com/cloudstack-software-engineering/>
> CloudStack Infrastructure Support<
> http://shapeblue.com/cloudstack-infrastructure-support/> | CloudStack
> Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message