incubator-cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Huang <Alex.Hu...@citrix.com>
Subject RE: [DISCUSS] PaaS Enablement: Composite Application Blueprints (#576)
Date Fri, 18 Jan 2013 01:07:07 GMT
I don't think this is off-topic at all.  The flip side of ingesting a blueprint is how to create
it.  Exporting a project is very interesting idea.

--Alex

> -----Original Message-----
> From: Kelcey Damage (BT) [mailto:kelcey@backbonetechnology.com] On
> Behalf Of kdamage@apache.org
> Sent: Thursday, January 17, 2013 10:24 AM
> To: cloudstack-dev@incubator.apache.org
> Subject: RE: [DISCUSS] PaaS Enablement: Composite Application Blueprints
> (#576)
>
> My comment about the projects was that as they stand, they are no
> recoverable. When you start a project you must expect it to terminate. There
> is no easy method to take the work you accomplish in a project and move it
> to a production system.
>
> It would be nice to be able to pack up a whole project in a vApp (Or at
> least and 'administrative migration' option, to migrate VM through admin
> boundaries)
>
> But I digress as this discussion is not about the projects themselves.
>
> -kd
>
>
> >-----Original Message-----
> >From: Jie Feng [mailto:Jie.Feng@citrix.com]
> >Sent: Thursday, January 17, 2013 9:47 AM
> >To: cloudstack-dev@incubator.apache.org
> >Subject: RE: [DISCUSS] PaaS Enablement: Composite Application Blueprints
> >(#576)
> >
> >Great analysis Chris!
> >
> >In terms of the blueprint model, I am also in favor of something simple.
> >CloudFormation can do what vApp is doing, and more. We only need to
> >implement a subset as a starter. For example, allow users to describe
> multiple
> >VMs, template ID for the VMs, and default service offering/disk offering,
> >networking for the interested zones, and startup order. Agree with Kelcey
> >that it does not need to be self-contained. We can support
> >importing/exporting of the blueprint to vApp (OVA/OVF) which contains
> both
> >the OVF descriptor and the template image files as an add-on feature.
> >
> >Do we need to map blueprints to project?  I thought project is more like a
> >shared workspace. IMO, we should have the flexibility to deploy multiple
> >blueprints and any VMs to a project.
> >
> >Jie
> >
> >> -----Original Message-----
> >> From: Kelceydamage@bbits [mailto:kelcey@bbits.ca]
> >> Sent: Thursday, January 17, 2013 1:51 AM
> >> To: cloudstack-dev@incubator.apache.org
> >> Subject: Re: [DISCUSS] PaaS Enablement: Composite Application
> >> Blueprints
> >> (#576)
> >>
> >> This was a good read, I think the simpler vApp method makes sense for
> >> an infrastructure tool. And could be a great opportunity to rework
> >> projects into a more useful system.
> >>
> >> I would not want to make the blueprints too complex, but my main issue
> >> with vApp methodology is how a vApp is completely self-contained. They
> >> eat up storage like mad, when they should really just be a collection
> >> of existing provisioning resources and templates.
> >>
> >> Example:
> >>
> >> A vApp when created copies the included VM temples to create a larger
> >> template. It would be better for our blueprints to simply point to the
> >> included templates like 'symlinks'. Providing almost zero increase to
> >> storage(only meta-data) and making the feature seriously competitive.
> >>
> >> The main logistics issue would be ensuring a blueprint health check
> >> incase someone deletes the linked templates underneath. Or maybe a
> >> dependancy error when someone tries to remove a template that's
> linked
> >> to one or more blueprints. This error could inform the admin that the
> >> template is in use by a blueprint that must be deleted first.
> >>
> >> Thanks
> >>
> >> Sent from my iPhone
> >>
> >> On Jan 16, 2013, at 11:51 PM, Chris Sears <chris.x.sears@sungard.com>
> >> wrote:
> >>
> >> > After reading up a bit more on AWS CloudFormation [1] and
> >> > OpenStack's work-alike Heat [2], they actually differ quite a bit
> >> > from VMware vApps. I thought it might be worth talking about some of
> >the differences.
> >> >
> >> > CloudFormation is a general purpose provisioning system for AWS
> >> > services, but under the hood, it's essentially a workflow engine
> >> > (similar to vCenter Orchestrator or HP Operations Orchestrator). The
> >> > problem CloudFormation is solving is runbook automation, where you
> >> > might have a 20 step process for deploying an app into AWS that you
> >> > want to completely automate. It competes with tools like Puppet and
> >> > Chef
> >> (although they can work together).
> >> >
> >> > The templates used by CloudFormation are less config files and more
> >> > like scripts or XSLT. They include variables, user inputs, outputs,
> >> > flow control, and a small standard library of functions. [3]
> >> > Templates require testing and debugging. It's possible to have
> >> > run-time bugs related to a wait condition that cause a deadlock and
> trigger
> >a rollback.
> >> >
> >> > When you execute a CloudFormation template, the resulting set of
> >> > provisioned resources is called a "stack". A stack could be a
> >> > multi-tier app with several VMs, but it could also be 3 empty VPCs,
> >> > an SNS queue, and an IAM user. If we follow a similar convention, a
> >> > stack would not necessarily map onto a project. A stack might
> >> > include VMs in multiple projects. Or no VMs at all.
> >> >
> >> > To extend the workflow coordination and configuration capabilities
> >> > into the guest OS, CloudFormation provides a set of helper scripts
> >> > that can fill a similar roll as cloud-init, creating files and
> >> > running commands with content and arguments dynamically generated
> >> > from the template. [4]
> >> >
> >> > It helped me to look at some real templates to understand everything
> >> > that CloudFormation was doing. Here are a couple examples... (the
> >> > first provisions an Active Directory server, the second creates an
> >> > instance of
> >> > OpenShift)
> >> > https://s3.amazonaws.com/cloudformation-templates-us-east-
> >> 1/Windows_Si
> >> > ngle_Server_Active_Directory.template
> >> >
> >>
> >https://github.com/openstack/heat/blob/master/templates/OpenShift.te
> m
> >> p
> >> > late
> >> >
> >> > By contrast, vApps are much simpler. They are effectively just a
> >> > collection of one or more VMs, their associated network settings and
> >> > some limited additional metadata. You can start and stop a vApp. You
> >> > can import or export a vApp as an OVF file if you wanted to move it
> >> > from one cloud to another. A vApp could roughly map to a project.
> >> >
> >> > To me, it sounds like we need vApps, with some limited features of
> >> > CloudFormation. Just enough to enable a user experience of clicking
> >> > a "Launch Blueprint" button and having everything magically
> >> > provisioned, like AWS does here:
> >> > http://aws.amazon.com/cloudformation/aws-cloudformation-
> templates/
> >> >
> >> > Thoughts?
> >> >
> >> > - Chris
> >> >
> >> > [1]: http://aws.amazon.com/cloudformation/faqs/
> >> > [2]: https://github.com/openstack/heat
> >> > [3]:
> >> >
> >>
> >http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/tem
> pl
> >> ate
> >> > -anatomy.html
> >> > [4]:
> >> >
> >http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-
> >> help
> >> > er-scripts-reference.html
> >> >
> >> >
> >> > On Wed, Jan 16, 2013 at 1:59 PM, Mohammad Nour El-Din <
> >> > nour.mohammad@gmail.com> wrote:
> >> >
> >> >> Hi
> >> >>
> >> >> Sent from my Samsung Galaxy S3
> >> >> Apologies for any typos
> >> >> On Jan 16, 2013 6:01 PM, "Shane Witbeck"
> <shane@digitalsanctum.com>
> >> wrote:
> >> >>>
> >> >>> +1 for YAML. After all, YAML is a superset of JSON [1] which is
> >> >>> +the
> >> >> primary objective of the 1.2 spec.
> >> >>>
> >> >>> Here are my pros and cons:
> >> >>>
> >> >>> Pros:
> >> >>> - relational anchors and aliases (potential here for less
> >> >>> verbosity
> >> >> through reuse)
> >> >>> - more readable
> >> >>> - comments
> >> >>> - projects which I respect, choose yaml over json for similar
> >> >>> types of
> >> >> configuration [2], [3]
> >> >>>
> >> >>> Cons:
> >> >>> - more complex to parse, less mature libraries compared to json
> >> >>>
> >> >>> All this being said, I would advocate that more initial focus
> >> >>> should be
> >> >> on the "model" and how it solves the problem of defining the
> >"blueprint".
> >> >> Serialization format discussion should be secondary and around
> >> >> whatever best fits the model, not the other way around.
> >> >>
> >> >> +1 on that
> >> >>
> >> >>>
> >> >>>
> >> >>> [1] http://yaml.org/spec/1.2/spec.html#id2759572
> >> >>> [2]
> >> >>
> http://www.elasticsearch.org/guide/reference/setup/configuration.ht
> >> >> ml
> >> >>> [3]
> >> >> https://github.com/cloudfoundry/bosh-
> >> release/blob/master/micro/aws.ym
> >> >> l
> >> >>>
> >> >>>
> >> >>> --
> >> >>> Shane Witbeck
> >> >>>
> >> >>>
> >> >>> On Thursday, January 10, 2013 at 1:20 PM, Kelcey Damage (BT) wrote:
> >> >>>
> >> >>>> +1 for YAML, I find JSON good for when I never look at the
data
> >> >>>> +and
> >> >> simply pass to a Python dictionary, however for interaction I am
> >> >> going to back the YAML choice.
> >> >>>>
> >> >>>> KELCEY DAMAGE
> >> >>>> Infrastructure Systems Architect
> >> >>>> www.backbonetechnology.com
> >> (http://www.backbonetechnology.com)
> >> >>>>
> >> >> -------------------------------------------------------------------
> >> >> --
> >> >> ----
> >> >>>> kelcey@backbonetechnology.com
> >> >>>> (mailto:kelcey@backbonetechnology.com)
> >> >>>>
> >> >>>> address: 55 East 7th Ave, Vancouver, BC, V5T 1M4
> >> >>>> tel: +1 604 713 8560 ext:114
> >> >>>> fax: +1 604 605 0964
> >> >>>> skype: kelcey.damage
> >> >>>>
> >> >>>>
> >> >>>>
> >> >>>>> -----Original Message-----
> >> >>>>> From: Alex Heneveld [mailto:alex.heneveld@cloudsoftcorp.com]
> >> >>>>> Sent: Wednesday, January 09, 2013 4:39 PM
> >> >>>>> To: cloudstack-dev@incubator.apache.org (mailto:
> >> >> cloudstack-dev@incubator.apache.org)
> >> >>>>> Cc: Min Chen
> >> >>>>> Subject: Re: [DISCUSS] PaaS Enablement: Composite Application
> >> >> Blueprints
> >> >>>>> (#576)
> >> >>>>>
> >> >>>>>
> >> >>>>> Hi Min, Jie,
> >> >>>>>
> >> >>>>> Min, nice questions. I've given my answers in-line.
> >> >>>>>
> >> >>>>> Jie, my answers also respond to some of your points, re
TOSCA
> >> >>>>> and YAML/JSON.
> >> >>>>>
> >> >>>>>
> >> >>>>> On 08/01/2013 17:50, Min Chen wrote:
> >> >>>>>> +1 on this feature to extend CloudStack from pure IaaS
to PaaS
> >> >>>>>> +or
> >> >> SaaS
> >> >>>>>> area. Some questions in my mind:
> >> >>>>>>
> >> >>>>>> 1) Does your proposed blueprint only contain functional
> >> >>>>>> component description or contain both functional and
> deployment
> >aspects?
> >> >>>>>> For example, for a 3-tier app containing three functional
> >> >>>>>> components
> >> >> like
> >> >>>>>> web tier, server, and DB, users can define different
deployment
> >> >> models
> >> >>>>>> for the same functional components, like small deployment
> >> >> (co-hosting
> >> >>>>>> all components on one VM), or medium or large deployment.
> >> >>>>>>
> >> >>>>>
> >> >>>>> Great use case. Introduces a _lot_ of complexity:
> >> >>>>>
> >> >>>>> (1) allow parameters to be passed to the blueprint (e.g.
"size")
> >> >>>>> (2) separate the components that are to be created from
the
> >> >> customizations
> >> >>>>> performed
> >> >>>>> (so you could define customizations for webserver, appserver,
> >> >>>>> and
> >> >> data,
> >> >>>>> then either
> >> >>>>> apply the customizations either to a single VM (small),
distinct
> >> >>>>> VM's (medium), or distinct VM groups (large))
> >> >>>>> (3) allow blueprints to refer to other blueprints (so the
> >> >>>>> "webserver customization" lives as its own reusable module)
> >> >>>>> (4) allow conditional branching based on parameters (e.g.
if
> >> >>>>> user selects "size=small" then run all customisations on
> >> >> single VM)
> >> >>>>>
> >> >>>>> I'd be inclined to DEFER them as features, until phase
2 or 3,
> >> >>>>> but we
> >> >> should
> >> >>>>> consider how the description might support them as I'd
like to
> >> >>>>> see
> >> >> some of
> >> >>>>> this functionality eventually.
> >> >>>>>
> >> >>>>> What do other people think? This is obvious functionality
to
> >> >>>>> want,
> >> >> what's the
> >> >>>>> right way to support it but without making the description
> >> >>>>> language complicated?
> >> >>>>>
> >> >>>>> (BTW one of the reasons I think TOSCA could be a good choice
is
> >> >>>>> that
> >> >> it has
> >> >>>>> considered many of these. You can define properties (1
above),
> >> >> arbitrary
> >> >>>>> nodes (2), and references to other definitions (3). It
also has
> >> >>>>> the
> >> >> concepts of
> >> >>>>> requirements / capabilities, relationships, policies, and
plans
> >> >>>>> --
> >> >> some of which
> >> >>>>> could be used to support (4)
> >> >>>>> eventually.)
> >> >>>>>
> >> >>>>>> 2) Have you thought of creating Service offerings from
your
> >> >>>>>> defined blueprints? This may allow CloudStack to provide
some
> >> >>>>>> SaaS
> >> >> functionalities.
> >> >>>>>>
> >> >>>>>
> >> >>>>> I like this idea, although it might break assumptions made
> >> >>>>> elsewhere
> >> >> about
> >> >>>>> service offerings so we should look at it carefully.
> >> >>>>>
> >> >>>>>> 3) As for blueprint description language, easy-to-read
and
> >> >>>>>> easy-to-edit by human being should be necessary if
we don't
> >> >>>>>> provide any visual GUI tools to create/edit a blueprint.
I have
> >> >>>>>> seen some
> >> >> JSON
> >> >>>>>> format before in BMC Cloud LifeCycle product 2.0.
> >> >>>>>>
> >> >>>>>
> >> >>>>> Wow JSON seems popular so far. And yet YAML is more concise
> and
> >> >>>>> expressive. Designed to be easy for people to read and
write
> >> >> configuration
> >> >>>>> just like this. Whereas JSON is designed for serializing
> >> >>>>> objects, in
> >> >> a way that
> >> >>>>> isn't too hard for people. YAML seems the right language
for
> >> >>>>> this
> >> >> purpose to
> >> >>>>> me -- and my experience has been that it is easier to read
and
> >> >>>>> to
> >> >> write,
> >> >>>>> without the litter of curly braces and quotes.
> >> >>>>>
> >> >>>>> But I don't want to be a language bigot! I can go with
JSON if
> >> >>>>> that's
> >> >> what
> >> >>>>> people want. :)
> >> >>>>>
> >> >>>>>> 4) For a multi-tier application, blueprint should not
only
> >> >>>>>> describe different components, but also connections
among
> those
> >> >>>>>> components, like open port, LB and FW rules among them.
I
> >> >>>>>> assume that your blueprint is also taking that into
> >> >>>>>> consideration, and your backend orchestration layer
can provide
> >> >>>>>> enough flexibility to provision any such app, seems
not an easy
> >> >>>>>> task to me since I am
> >> new to CS.
> >> >>>>>>
> >> >>>>>
> >> >>>>> Yeah, this is where it gets interesting. A common trick
I've
> >> >>>>> seen is
> >> >> to use
> >> >>>>> hostnames for a lot of the connection configuration, and
have a
> >> >>>>> few
> >> >> hard-
> >> >>>>> coded patterns in there (e.g. for groups, load-balancers).
But
> >> >>>>> that
> >> >> only gets
> >> >>>>> you so far, and then you hit a wall (or spin up a different
> >> >> orchestrator).
> >> >>>>>
> >> >>>>> Using requirements/capabilities and typed relationships
gives a
> >> >>>>> much
> >> >> more an
> >> >>>>> elegant solution. (Not necessarily TOSCA, but these concepts
are
> >> >>>>> well developed there!)
> >> >>>>>
> >> >>>>> In any case let's make sure the use cases include interesting
> >> >> situations like
> >> >>>>> this, and we can compare approaches. This absolutely DOES
need
> >> >>>>> to be
> >> >> in
> >> >>>>> phase 1 I think.
> >> >>>>>
> >> >>>>>> 5) From a normal user perspective, I would really love
to see a
> >> >>>>>> GUI tool developed around this to allow user to visually
> >> >>>>>> create/edit a multi-tier app blueprint instead of using
a
> TextEditor.
> >> >>>>>>
> >> >>>>>
> >> >>>>> Nice to hear. Me too. The description, API, and backend
come
> >> >>>>> first,
> >> >> with an
> >> >>>>> initial set of features, but I'd love to see the GUI in
phase 2.
> >> >>>>>
> >> >>>>>> Thanks
> >> >>>>>> -min
> >> >>>>>>
> >> >>>>>
> >> >>>>> Keep it coming please.
> >> >>>>>
> >> >>>>> --Alex
> >> >>>
> >> >>


Mime
View raw message