cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rohit Yadav <rohit.ya...@shapeblue.com>
Subject Re: [DISCUSS] 4.6 release management
Date Wed, 29 Apr 2015 15:54:49 GMT
Hi Remi,

Thanks. Sure we can work together on this, I guess you would be running KVM/XenServer on KVM.
Ping me if you need help on running ESX 5.x on KVM as it requires a patched qemu system binary
(prebuilt debs here http://people.apache.org/~bhaisaab/qemu). If these nested hosts are managed
by CloudStack itself instead of virsh or virt-viewer, then use ACS 4.5 or master.

> On 29-Apr-2015, at 5:21 pm, Remi Bergsma <RBergsma@schubergphilis.com> wrote:
>
> Hi Rohit,
>
> Nice work!
>
> I agree we need an environment that does run on something else than the local machine,
as we need more horse power. We worked on something similar, where we have a cluster of KVM
controlled by CloudStack in our Employee Cloud and spin large VMs running CentOS 7.1 (we use
32 or 64GB ram). In this big VM we run KVM again and spin up all infrastructure, ranging from
hypervisors, SDN controllers, database hosts, storage appliances, etc. The big benefit is
that networking is easy, as it is all in one box. Also, everybody uses the same prefixes.
Using a L2 VPN we connect the workstations to the dev/test environment. All VMs can be controlled
using virt-manager (like you also showed) and it is actually CloudStack running inside CloudStack.
Deploying this VM has been automated and we’re now working on automating different scenario’s.
They can also be used as Jenkins build slaves, for example. It is work in progress.
>
> Long story short: Let's work together on this :-)
>
> Regards,
> Remi
>
>
>> On 28 Apr 2015, at 21:07, Rohit Yadav <rohit.yadav@shapeblue.com> wrote:
>>
>> Hi Ilya,
>>
>> In short - to run ESX and other hypervisors (Xen, KVM, OVM3, HyperV etc) on KVM you
need to;
>>
>> - use patched qemu (tested to work on both Ubuntu 14.04 and 15.04 x64, I’m waiting
for Fedora 22 to test it on F22 as well), you may install the pre-built debs or build/install
qemu from source using the patch from here: people.apache.org/~bhaisaab/qemu
>> - use ACS using latest 4.5 branch and deploy a basic zone (without SG) to provision
hypervisor hosts as user vms
>> - in agent.properties (on your kvm host), enable guest.cpu.features=vmx and guest.cpu.mode=host-passthrough
>> - when deploying ESX 5.x on vm (or installing using an ISO) deploy virtualmachine
with details nicAdapter=vmxnet3, for other hypervisors (including ESX 6.0) E1000, the default
nic adapter, works
>>
>> IMO this is a better approach as it does not depend on ESX or VMWare fusion that
requires special hardware (vCenter/Windows etc or OSX/Apple machine) and are difficult to
automate. Working with KVM host, since is a Linux machine, would be more familiar to sysadmins
and certainly a pleasure to scale and work with because one can avoid management tools (such
as XenCenter or vCenter).
>>
>> (I’m still working on the "CloudStack Developer Kit” {CDK} so it’s not in a
state to be released yet, will avoid publishing a faulty tool now. Instead of the DevCloud
approach which promotes everything on one machine, the CDK I’m trying to build focuses on
developer productivity, reproducibility and scalability of a QA lab, it recommends at least
one companion hardware with a developer’s workstation/laptop which can be a small-form-factor
server like a mini PC or NUC with at least 16GB RAM and 4 cores i7 with Intel VT. Hope to
share it soon.)
>>
>>> On 28-Apr-2015, at 7:17 pm, ilya <ilya.mailing.lists@gmail.com> wrote:
>>>
>>> Rohit
>>>
>>> Any headway on ESX 5.5? I've done this many times before using cloudstack and
esx, but i was using esx as parent hypervisor.
>>>
>>> The challenge for me was being able to automatically deploy and configure the
vSphere + ESXi env. I managed to get the whole flow working with bash script, puppet, VMA
and while it works its not pretty. The challenge was the networking bit.
>>>
>>> Last but not least, consider using cloudstack to test cloudstack. i.e. master
env, would use cloudstack projects and spinup smaller envs with KVM, Xen and VmWare bound
to each project.
>>>
>>> Regards
>>> ilya
>>>
>>> On 4/24/15 7:53 AM, Rohit Yadav wrote:
>>>> Daan,
>>>>
>>>>> On 24-Apr-2015, at 3:53 pm, Daan Hoogland <daan.hoogland@gmail.com>
wrote:
>>>>>
>>>>> Rohit, the issues you mention are not as painful if we release in a
>>>>> two week schedule as the period of creating a fix to seeing it in a
>>>>> release will be shorter. Some releases will be broken for some people,
>>>>> I don't think we can prevent this. The target we are aiming for is to
>>>>> big to cover it completely.
>>>> I agree with you, but I think there are pros and cons to both approaches
and for this to work it needs to be able to walk first before it can run.
>>>>
>>>> For this to work we need an automated QA system, to solve this Abhi is working
on it for past few weeks and will be adding more non-hardware tests (simulator ones) to travis.
In my free time, I’m trying to setup a nested virtualized environment where we can test
ACS against Xen, KVM and VMware on top on KVM. So far, I’m able to run XenServer 6.2+6.5
and KVM on top of KVM with vmx (intel-vt) enabled, and making some progress with running ESX
on KVM (I’m able to run ESX 6.0 on KVM now, but not ESX 5.x which is something I’m exploring).
I hope we'll have something working soon that is fairly fast and easy to reproduce.
>>>>
>>>>> Your points are valid, though.
>>>>> .1 a three person release team makes sense. I have been really happy
>>>>> with the help I got from Pierre-Luc and I think David can do with help
>>>>> the coming time as well.
>>>>> .2 Hopefully people won't need to test every release so extensively
>>>>> anymore as the changes become smaller. (and my initial remark the
>>>>> above applies as well)
>>>> By having too many releases we’ll have to deal with too many upgrade path
issues and users will spread across different versions which will create an issue for maintainers
who are supporting users -- one solution for this problem can be that we introduce a concept
of LTS release that is either maintained by the community (which can be difficult) or some
interested stakeholders, for users that would prefer upgrading everytime there is a new CloudStack
release.
>>>>
>>>>> On Fri, Apr 24, 2015 at 2:43 PM, Rohit Yadav <rohit.yadav@shapeblue.com>
wrote:
>>>>>> I think we need to have a faster release management to speed up process
in general, and for that I propose that we have at least two co-pilots for the release manager
who would support them with things like reviewing/merging patches, creating RC candidates
etc whenever necessary. Having only one person as a release manager can become a bottleneck
for a speedy release.
>>>>>>
>>>>>> The other issue is getting people to test a (release) branch, fix
bugs and expect a review/result in 72 hours. This has usually failed if people are busy and
not getting enough time for this. As an example, I think 4.5 is delayed because it lacked
people actively testing it or fixing issues, or when issues were found only around the RC
testing period which delayed RC voting by 1-2 weeks every time that happened. (I’ll post
details about where I think we are wrt 4.5 in another thread).
>>>>>>
>>>>>>> On 17-Apr-2015, at 12:49 am, Pierre-Luc Dion <pdion@cloudops.com>
wrote:
>>>>>>>
>>>>>>> Today during the CloudStackdays  we did a round table about Release
>>>>>>> management targeting the next 4.6 releases.
>>>>>>>
>>>>>>>
>>>>>>> Quick bullet point discussions:
>>>>>>>
>>>>>>> ideas to change release planning
>>>>>>>
>>>>>>> - Plugin contribution is complicated because often  a new plugin
involve
>>>>>>> change on the core:
>>>>>>>   - ex: storage plugin involve changes on Hypervisor code
>>>>>>> - There is an idea of going on a 2 weeks release model which
could
>>>>>>> introduce issue the database schema.
>>>>>>> - Database schema version should be different then the application
>>>>>>> version.
>>>>>>> - There is a will to enforce git workflow in 4.6  and trigger
simulator
>>>>>>> job on  PullRequest.
>>>>>>> - Some people (I'm part of them) are concerned on our current
way of
>>>>>>> supporting and back porting fixes to multiple release (4.3.x,
4.4.x,
>>>>>>> 4.5.x). But the current level of confidence against latest release
is low,
>>>>>>> so that need to be improved.
>>>>>>>
>>>>>>>
>>>>>>> So, the main messages is that w'd like to improve the release
velocity, and
>>>>>>> release branch stability.  so we would like to propose few change
in the
>>>>>>> way we would add code to the 4.6 branch as follow:
>>>>>>>
>>>>>>> - All new contribution to 4.6 would be thru Pull Request or merge
request,
>>>>>>> which would trigger a simulator job, ideally only if that pass
the PR would
>>>>>>> be accepted and automatically merged.  At this time, I think
we pretty much
>>>>>>> have everything in place to do that. At a first step we would
use
>>>>>>> simulator+marvin jobs then improve tests coverage from there.
>>>>>>>
>>>>>>> Please comments :-)
>>>>>> Regards,
>>>>>> Rohit Yadav
>>>>>> Software Architect, ShapeBlue
>>>>>> M. +91 88 262 30892 | rohit.yadav@shapeblue.com
>>>>>> Blog: bhaisaab.org | Twitter: @_bhaisaab
>>>>>>
>>>>>>
>>>>>>
>>>>>> Find out more about ShapeBlue and our range of CloudStack related
services
>>>>>>
>>>>>> IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//>
>>>>>> CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
>>>>>> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
>>>>>> CloudStack Software Engineering<http://shapeblue.com/cloudstack-software-engineering/>
>>>>>> CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/>
>>>>>> CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
>>>>>>
>>>>>> This email and any attachments to it may be confidential and are
intended solely for the use of the individual to whom it is addressed. Any views or opinions
expressed are solely those of the author and do not necessarily represent those of Shape Blue
Ltd or related companies. If you are not the intended recipient of this email, you must neither
take any action based upon its contents, nor copy or show it to anyone. Please contact the
sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated
in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and
is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company
incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty
Ltd is a company registered by The Republic of South Africa and is traded under license from
Shape Blue Ltd. ShapeBlue is a registered trademark.
>>>>>
>>>>>
>>>>> --
>>>>> Daan
>>>> Regards,
>>>> Rohit Yadav
>>>> Software Architect, ShapeBlue
>>>> M. +91 88 262 30892 | rohit.yadav@shapeblue.com
>>>> Blog: bhaisaab.org | Twitter: @_bhaisaab
>>>>
>>>>
>>>>
>>>> Find out more about ShapeBlue and our range of CloudStack related services
>>>>
>>>> IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//>
>>>> CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
>>>> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
>>>> CloudStack Software Engineering<http://shapeblue.com/cloudstack-software-engineering/>
>>>> CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/>
>>>> CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
>>>>
>>>> This email and any attachments to it may be confidential and are intended
solely for the use of the individual to whom it is addressed. Any views or opinions expressed
are solely those of the author and do not necessarily represent those of Shape Blue Ltd or
related companies. If you are not the intended recipient of this email, you must neither take
any action based upon its contents, nor copy or show it to anyone. Please contact the sender
if you believe you have received this email in error. Shape Blue Ltd is a company incorporated
in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and
is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company
incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty
Ltd is a company registered by The Republic of South Africa and is traded under license from
Shape Blue Ltd. ShapeBlue is a registered trademark.
>>>
>>>
>>
>> Regards,
>> Rohit Yadav
>> Software Architect, ShapeBlue
>> M. +91 88 262 30892 | rohit.yadav@shapeblue.com
>> Blog: bhaisaab.org | Twitter: @_bhaisaab
>>
>>
>>
>> Find out more about ShapeBlue and our range of CloudStack related services
>>
>> IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//>
>> CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
>> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
>> CloudStack Software Engineering<http://shapeblue.com/cloudstack-software-engineering/>
>> CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/>
>> CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
>>
>> This email and any attachments to it may be confidential and are intended solely
for the use of the individual to whom it is addressed. Any views or opinions expressed are
solely those of the author and do not necessarily represent those of Shape Blue Ltd or related
companies. If you are not the intended recipient of this email, you must neither take any
action based upon its contents, nor copy or show it to anyone. Please contact the sender if
you believe you have received this email in error. Shape Blue Ltd is a company incorporated
in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and
is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company
incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty
Ltd is a company registered by The Republic of South Africa and is traded under license from
Shape Blue Ltd. ShapeBlue is a registered trademark.
>

Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +91 88 262 30892 | rohit.yadav@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab



Find out more about ShapeBlue and our range of CloudStack related services

IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//>
CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
CloudStack Software Engineering<http://shapeblue.com/cloudstack-software-engineering/>
CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/>
CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>

This email and any attachments to it may be confidential and are intended solely for the use
of the individual to whom it is addressed. Any views or opinions expressed are solely those
of the author and do not necessarily represent those of Shape Blue Ltd or related companies.
If you are not the intended recipient of this email, you must neither take any action based
upon its contents, nor copy or show it to anyone. Please contact the sender if you believe
you have received this email in error. Shape Blue Ltd is a company incorporated in England
& Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated
under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated
in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company
registered by The Republic of South Africa and is traded under license from Shape Blue Ltd.
ShapeBlue is a registered trademark.
Mime
View raw message