cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Youcef Laribi <Youcef.Lar...@eu.citrix.com>
Subject RE: AutoScale feature....
Date Tue, 03 Jul 2012 19:11:59 GMT

>>>It is not necessary for an LB to implement the auto scaling. Any other 
>>>component can do the same. For example, a component can monitor via 
>>>hypervisor stats, or volume IO stats. So in >this sense it is not 
>>>generic.
>>
>>It is not necessary for an LB to implement the auto scaling but 
>>NetScaler LB provides one and we want to allow CS admins who are using 
>>NetScaler to be able to take advantage of it. There is nothing here 
>>that prevents other components to take advantage of Counters, AutoScale policies,
etc.
>>if they wish to.
>
>I don't think the point was about "preventing other components". It is around enabling
other implementations.
>The question was - is there a generic framework with the Netscaler implementation being
one exemplar or instantiation of the framework.
> If so, we would want the FS and API to be structured around the framework first and detail
how the Netscaler implementation realizes (extends / implements/plugs in) to the framework.

If by enabling other implementations we mean that the autoscaling infrastructure is fully
implemented inside CloudStack (the monitoring of the VMs, the triggering the scale events,
etc.) and doesn't assume/require loadbalancers or other components to have this functionality,
this is *not* what this proposal is about.  

This proposal is about *exposing* a capability that some loadbalancers like NetScaler have,
and should be seen in the "LB" context.

The question then is: do we wait until CloudStack provides a monitoring/scaling infrastructure
(which is a much more ambitious task) and therefore can be used on any loadbalancer not just
the ones that implement this capability (therefore neutralizing any advantage a loadbalancer
has over another), or do we allow LB devices that already have this capability to be exposed
to CS users? IMO, the two are legitimate contributions. As an LB vendor, what we are contributing
here is to the second proposal. 

Youcef


-----Original Message-----
From: Chiradeep Vittal [mailto:Chiradeep.Vittal@citrix.com] 
Sent: Tuesday, July 03, 2012 10:50 AM
To: CloudStack DeveloperList
Subject: Re: AutoScale feature....



On 7/2/12 11:43 PM, "Youcef Laribi" <Youcef.Laribi@eu.citrix.com> wrote:

>In a nutshell, the framework is as follows:
>
>
>
>1.       VM Counters that specify the monitored aspects of the VMs (some
>of these counters are monitored through SNMP agents running inside the 
>VM
>- hence they are of the SNMP type)

This is specific to the implementation. There are several ways to monitor performance. SNMP
is just one. Hence it should not be part of the API. It can be a generic key-value pair (for
an example, look at the createLbStickinessPolicy http://download.cloud.com/releases/3.0.0/api_3.0.0/user/createLBStickinessP
olicy.html)

>
>2.       AutoScale Policy to express the low and high watermark
>thresholds on counters to trigger a scaling up/down action.

This is generic

>
>3.       VM Profile that contains the details needed to auto-instantiates
>VMs and captures anything VM-specific (template, service offering, 
>network, VM's SNMP agent port and community, etc.)

Template, service offering and network are generic, SNMP is not.

>
>4.       AutoScale Group which represents the dynamic list of autoscale
>VMs (min/max members in this group), and associates with them an 
>autoscale policy  and a VM profile. Once defined, the AutoScale Group 
>is assigned to the LB Rule (just like assigning manually one 
>VM/instance to an LB Rule).

This is generic
>
>
>
>This is a generic framework.
>
>
>
>The implementation relies on the LoadBalancer Resource being able to 
>support this new config item in an LB Rule. Any LB offering in 
>CloudStack can support this new additional config and is free in the 
>way it implements it. CloudStack will only allow configuring this 
>feature on an LB Rule when the network LB offering supports it. The 
>NetScaler offering will support this new feature, and we are extending 
>the existing NetScaler Resource layer to implement it.
>

It is not necessary for an LB to implement the auto scaling. Any other component can do the
same. For example, a component can monitor via hypervisor stats, or volume IO stats. So in
this sense it is not generic.

>
>
>Youcef

--
Chiradeep
>


-----Original Message-----
From: Youcef Laribi [mailto:Youcef.Laribi@eu.citrix.com] 
Sent: Monday, July 02, 2012 11:43 PM
To: cloudstack-dev@incubator.apache.org
Subject: Re: AutoScale feature....

In a nutshell, the framework is as follows:



1.       VM Counters that specify the monitored aspects of the VMs (some of these counters
are monitored through SNMP agents running inside the VM - hence they are of the SNMP type)

2.       AutoScale Policy to express the low and high watermark thresholds on counters to
trigger a scaling up/down action.

3.       VM Profile that contains the details needed to auto-instantiates VMs and captures
anything VM-specific (template, service offering, network, VM's SNMP agent port and community,
etc.)

4.       AutoScale Group which represents the dynamic list of autoscale VMs (min/max members
in this group), and associates with them an autoscale policy  and a VM profile. Once defined,
the AutoScale Group is assigned to the LB Rule (just like assigning manually one VM/instance
to an LB Rule).



This is a generic framework.



The implementation relies on the LoadBalancer Resource being able to support this new config
item in an LB Rule. Any LB offering in CloudStack can support this new additional config and
is free in the way it implements it. CloudStack will only allow configuring this feature on
an LB Rule when the network LB offering supports it. The NetScaler offering will support this
new feature, and we are extending the existing NetScaler Resource layer to implement it.



Youcef





>On 7/2/12 6:24 PM, "David Nalley" <david@gnsa.us<mailto:david@gnsa.us>> wrote:

>

>>On Mon, Jul 2, 2012 at 2:08 PM, Ram Ganesh <Ram.Ganesh@citrix.com<mailto:Ram.Ganesh@citrix.com>>
wrote:

>>> Hi All,

>>>

>>> Briefly talking about the feature - AutoScale feature allows you to

>>>scale up or scale down the virtual instances that are being load

>>>balanced by load balancers like NetScaler based on certain conditions.

>>>This feature is an extension to the ELB feature in CloudStack. This

>>>feature is similar to AWS autoscale feature except that monitoring of

>>>server's(guest vm) health is done by load balancers like NetScaler and

>>>provisioning is done by CloudStack. Monitoring done by loadbalancers

>>>has couple of benefits

>>>         - Loadbalancers are better placed to understand the health of

>>>the  server as traditionally LB devices had monitoring capability

>>>inbuilt in it.

>>>         - Decisions can be based not just on certain conventional

>>>counters like CPU, Memory etc but also on network counters like

>>>response time, bandwidth, connections etc

>>>         - Decreases the load on CloudStack management server.

>>

>>But that inherently limits where AutoScale will work. This means all of

>>this work is inherently very niche, can only be tested by folks with NS

>>hardware,  and only utilized by the same folks - and autoscaling could

>>be inherently useful in other situations that don't have LB

>>implications.

>>CloudStack has inherently been hypervisor agnostic, is there a reason

>>not to be LB agnostic as well?

>

>It seems to me that the proposed APIs are not too LB-specific (there's stuff about snmp
ports and community strings that is specific to this implementation and should not belong
in a >generic API).

>I would imagine that an auto scaling implementation would have:

>1. Monitors that know how to monitor a specific kind of load 2. Configuration of thresholds
to get notified when the monitors detect a breach of thresholds 3. Actions to take on breach
>of thresholds

>

>I would like to see a generic framework such as this laid out and a description of how
this specific implementation uses the framework.

>

>Also, the FS template has lots of questions (http://wiki.cloudstack.org/display/RelOps/Functional+Spec%28FS%29+Template

>) which have not been answered.

>

Mime
View raw message