incubator-cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Huang <Alex.Hu...@citrix.com>
Subject RE: [DISCUSS] Affinity / Anti-affinity Rules
Date Wed, 06 Feb 2013 19:59:44 GMT
Prachi,

If it's an elective option, then it shouldn't have options in the deployvm api.  We can't
decouple the code but then say it's coupled at deployvm api call.

I think it makes sense to have the affinity be part of orchestration.  If so, then it makes
sense to do this.

- Write DeploymentPlanningManager which contains planning for volume, vm, and network.
- For VM deployment, DPM goes through the following stages
	- Affinity - to set deploymentplan scope and exclude list (we might think about merging the
two).
	- DeploymentPlanner - to use heuristics to find the best spot to place the vm/volume.
	- Allocator - which matches the requirements to capabilities of the physical resource.

If you think about it, then it makes sense because each matches to the functionality it serves.
	- Affinity - matches user preference to narrow down where to deploy.
	- DeploymentPlanner - matches algorithms set by Administrator to give best performance/value/feature
	- Allocator - matches physical requirements.

This would mean taking Allocator out of DeploymentPlanner implementations.  It is very similar
to your intent with an AffinityDeploymentPlanner.

I think actually this works out very well with the idea of reservations as well.  You might
think about doing the two together as a refactor of Deployment Planning.

--Alex

> -----Original Message-----
> From: Prachi Damle
> Sent: Tuesday, February 05, 2013 11:39 PM
> To: Alex Huang; cloudstack-dev@incubator.apache.org
> Subject: RE: [DISCUSS] Affinity / Anti-affinity Rules
> 
> Hi Alex,
> 
> Thanks for the review, I have answered inline. Please comment.
> I guess the FS needs more description reasoning the 2-component design to
> avoid confusion.
> 
> -Prachi
> 
> -----Original Message-----
> From: Alex Huang
> Sent: Tuesday, February 05, 2013 4:49 PM
> To: Prachi Damle; cloudstack-dev@incubator.apache.org
> Subject: RE: [DISCUSS] Affinity / Anti-affinity Rules
> 
> Hi Prachi,
> 
> A few comments about this spec.
> 
> - What is the error when planning fails?  What details will it give?
> 
> 
> [Prachi] I was planning to still rely on the deployment planner to throw
> exception since planner will be the component that will find out that
> placement is not possible.
> But it is a good point and I will add a custom exception that the
> AffinityProcessor can throw if upon analysis it finds the affinity rule cannot be
> followed. This will save the planner execution.
> 
> 
> - Why are HostAffinityProcessor and HostAntiAffinityProcessor different
> implementations?  The requirements says they should be specified
> concurrently so shouldn't they be the same processor?  Or are you planning
> for this adapter to be an adapter chain?
> 
> 
> [Prachi] Yes I am planning this to be a chain of adapters, each can handle a
> specific affinity type.
> 
> - I'm not really sure I understand the design.  It would seem like this can be
> designed in two ways.
> 	1. Affinity Group is an integrated part of cloud-engine and
> AffinityGroupProcessor is ran before all DeploymentPlanners to set the
> scope of DeploymentPlan and ExcludeList.  If this is the case, I don't
> understand why there is a AffinityGroupPlanner.
> 	2. The entire thing is implemented as a DeploymentPlanner.  If it is
> implemented as a DeploymentPlanner, then why do we need the
> AffinityGroupProcessor interface?  Why not just have it as a property of that
> DeploymentPlanner implementation?
> After giving it some thought, I think it is important that affinity group is part
> of core orchestration.  If so then I would prefer 1.  The current spec has both
> so I'm confused on this.
> - The getType() of the AffinityGroupProcessor seems like it's more similar to
> the getName of the adapter already, which needs to be unique.  I think you
> just need more descriptions on what the processor does so the end user can
> understand how to use it?
> 
> [Prachi] Since use of the affinity groups will be done during VM placement, I
> wanted to stick to add all logic to DeploymentPlanner. This will make sure
> that anytime the planner gets called (deployment, restart, migration, HA)
> affinity groups are take care of.  Still I see the need of being able to plugin
> custom implementations for various types of affinity/anti-affinity. So pushed
> out the handling of the affinityType to a separate adapter.
> 
> Thus AffinityPlanner could be as simple as just running through the chain of
> AffinityGroupProcessor's and it will not change even if we introduce any new
> affinity types. It will be the hook in Cloudstack, for the plugins implementing
> affinity types
> 
> As you say we could keep this part of cloud-engine and run through the chain
> of AffinityGroupProcessor before planners - but it will not take care of other
> usecases that directly call planners for placement.
> 
> - Can you check with Min on her creation of the REST API?  Can we start using
> the REST only API from here on?
> [Prachi ]Sure will sync up with Min on REST API.
> 
> --Alex
> 
> > -----Original Message-----
> > From: Prachi Damle [mailto:Prachi.Damle@citrix.com]
> > Sent: Tuesday, February 05, 2013 2:42 PM
> > To: cloudstack-dev@incubator.apache.org
> > Subject: RE: [DISCUSS] Affinity / Anti-affinity Rules
> >
> > Hey all,
> >
> > Following up on this feature discussion. I have put up an initial
> > draft of the FS for this feature here:
> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/FS+-+Affinity-
> > Anti-affinity+groups
> >
> > As per discussions below, the scope of the feature now consists of a
> > generic framework for defining affinity groups in CloudStack and a
> > default implementation to support host affinity and anti-affinity.
> >
> > Please provide your comments.
> >
> > Thanks,
> > Prachi
> >
> > -----Original Message-----
> > From: Chip Childers [mailto:chip.childers@sungard.com]
> > Sent: Monday, January 07, 2013 6:30 AM
> > To: cloudstack-dev@incubator.apache.org
> > Subject: Re: [DISCUSS] Affinity / Anti-affinity Rules
> >
> > On Mon, Jan 7, 2013 at 1:22 AM, Chris Sears
> > <chris.x.sears@sungard.com>
> > wrote:
> > > Greetings,
> > >
> > > I understand the motivation for a feature like this, but I'm
> > > concerned that the concepts of affinity and anti-affinity might not
> > > be appropriate cloud-level abstractions to expose to end users.
> > > Also, it might be difficult to effectively automate decisions about
> > > fault and performance domains, both of which would vary greatly
> > > between
> > deployments.
> > >
> > > Using anti-affinity rules to make sure HA-related VMs aren't placed
> > > on the same host seems like the most critical use case. What if we
> > > narrowed the scope of the feature to just address that issue?
> > > Building on Chiradeep's idea, VMs could have an anti-affinity group
> attribute.
> > > VMs in the same anti-affinity group must be placed on different hosts.
> > > For the first implementation, the guarantee would only apply to
> > > initial
> > provisioning.
> >
> > It could actually apply any time a planner is used to select a host
> > (which I think also includes CloudStack "HA").
> >
> > > @Manan, would that be sufficient?
> > >
> > > Regards
> > >
> > >  - Chris
> > >
> > >
> > > On Fri, Jan 4, 2013 at 6:50 PM, Prachi Damle
> > <Prachi.Damle@citrix.com>wrote:
> > >
> > >> Yes, requirements seem vague. What parameters define
> > >> affinity/anti-affinity?
> > >>
> > >> Requirements mention
> > >> >>  For each VM, users should be able to provide both (Affinity VMs
> > >> >> and
> > >> Anti-affinity VMs) lists concurrently. For example, VM-A can have
> > >> affinity with VMs B & C and anti-affinity with VMs D & E at the
same
> time.
> > >> >> When configuring Affinity / anti-affinity for a VM, users should
> > >> >> be
> > >> allowed to provide a list of affinity / anti-affinity VMs (via API)
> > >> or select affinity /anti-affinity VMs from a list (via UI)
> > >>
> > >> When user specifies VM-A can have affinity with VMs B & C does that
> > >> mean they should be placed on same pod or same hypervisor(cluster
> > >> or
> > >> host) by the allocation logic?
> > >>
> > >> -Prachi
> > >> -----Original Message-----
> > >> From: Chiradeep Vittal [mailto:Chiradeep.Vittal@citrix.com]
> > >> Sent: Thursday, January 03, 2013 6:06 PM
> > >> To: CloudStack DeveloperList
> > >> Subject: Re: [DISCUSS] Affinity / Anti-affinity Rules
> > >>
> > >> Actually the proposal is quite vague.
> > >> What does affinity mean to the end-user?
> > >> What guarantees are being asked for?
> > >>  - the vms are on the same hypervisor (affinity)
> > >>  - the vms are not on the same hypervisor (anti)
> > >>  - the vms are interconnected by a high-speed interconnect
> > >> (affinity)
> > >>  - the vms are in different failure domains (host/cluster/pod)
> > >>
> > >> I find the concept of affinity groups useful.
> > >> A possible workflow would be
> > >> 1. Create an affinity group of type 'Foo'
> > >> 1a. Group type indicates the guarantee 2. Create a VM in the group
> > >>
> > >> VMs can only leave groups on vm destruction.
> > >>
> > >> But without the specific type of guarantee, it is hard to discuss
> > >> this proposal.
> > >>
> > >> On 1/3/13 4:23 PM, "Manan Shah" <manan.shah@citrix.com> wrote:
> > >>
> > >> >Hi,
> > >> >
> > >> >I would like to propose a new feature for enabling Affinity /
> > >> >Anti-affinity rules in CS 4.1. I have created a JIRA ticket and
> > >> >provided the requirements at the following location.  Please
> > >> >provide feedback on the requirements.
> > >> >
> > >> >JIRA Ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-739
> > >> >Requirements:
> > >> >https://cwiki.apache.org/confluence/display/CLOUDSTACK/Affinity+-
> > +An
> > >> >ti-
> > >> >aff
> > >> >i
> > >> >nity+rules
> > >> >
> > >> >
> > >> >Regards,
> > >> >Manan Shah
> > >> >
> > >>
> > >>
> > >>

Mime
View raw message