stratos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shaheed Haque <>
Subject Re: [Discuss] Grouping of services (cartridges)
Date Thu, 01 May 2014 08:13:23 GMT

Sorry for top-posting, but one additional thing that I feel should be 
considered is a minimal set of lifecycle events for the group; for example:

- "all members of a group are now active", aka "group active".

- "at least one member of an existing group has failed", aka "group down".

This is clearly an add-on to the core functionality but is needed to round the 
feature out.

Thanks, Shaheed

On Thursday 01 May 2014 07:46:46 Lakmal Warusawithana wrote:

Hi Martin,

With the current implementation I think and agree its better to go with json 
as defining configuration data. Summarizing proposal and the discussion I can 
think following data model. Here I am thinking implementing single 
configuration data (application deployment definition) model, which can define 
full configuration of an composite application. we should take it down to 
several milestones. we can start on enhancing backend services in initial 
milestone and later will come with aggregating them. All of your thoughts 

Can you come up with some definition format. (may be json format). Also see my 
inline comment.

On Wed, Apr 30, 2014 at 11:03 PM, Martin Eppel (meppel) <> 

Hi Lakmal,
Below are ideas on how to break up the feature into a high level ToDo list 
(with some suggestions and questions) : 
+ Defining configuration data format – for example json or chef / recipe
   ++ would using recipes affect all configuration data or is it restricted to 
the grouping / dependency configuration only ?

IMHO, its may easy with json, considering current existing services.

+ Adding the new model to stratos manager and integrate with existing model:
   ++ Group model (nested groups, group properties)
    ++ Dependency model
    ++ in the current model we have a Subscriptions and Deployment manager, I 
think we need something similar to handle the groups which in turn invokes 
deployment / subscriptions manager

+ Defining runtime dependency conditions:
      ++ Calculating the dependency for a cartridge (service) dynamically by 
walking the dependency tree and checking the state of dependencies (e.g. 
member state)

+ Integrate the dynamic aspect of the dependency model (checking the 
conditions of dependencies and act accordingly):
      ++ As of now I think it has to be integrated with the autoscaler as it 
the entity which controls instances (members) and checks their states

Yes, we need to enhance auto scaler
      ++ How is dependency retrieved from configuration data (e.g. published 
through topology, beans, etc ….) ?

We need to think on more this. IMO, we may need another topic (application 
deployment definition) to retrieved the config data. SM can define this based 
on provide definition, auto scaler and who ever need can get that. Topology is 
maintaining current running state (currently its for a service, we need to 
enhance it to application), so we can used "application deployment definition" 
data to check with current state.
      ++ Integrate dependency checks into the runtime portion of the 
autoscaler, e.g.  add the checks to the monitor functionality of a 
ClusterMonitor with respective actions (spawning  / terminating instances)

I think we can coupled auto scaling policy with individual 
What do you think ?

We may need to think about deployment policy, I think with the application 
deployment definition (with some coupling ,like some cartridges may need to 
spin up same partition..etc). May be we are redefining deployment policy with 
application deployment definition. (we may obsolete deployment policy) 
From: Lakmal Warusawithana [] 
Sent: Wednesday, April 30, 2014 4:30 AM

Cc: Shaheedur Haque (shahhaqu)

Subject: Re: [Discuss] Grouping of services (cartridges)
Hi Martin,
These use cases are very valid, and we should integrate them into 4.1.0. Will 
go through in detail and see how we can incorporate into Stratos.
I have one question, why do we need hierarchical grouping, (I mean group 
inside a group) and use case? Can't we have flat?
On Wed, Apr 30, 2014 at 9:56 AM, Martin Eppel (meppel) <> 
Hi Lakmal
Below is a further enhanced proposal to add grouping to apache stratos:
In a nutshell:
By grouping cartridges together we can define characteristics which apply to 
all cartridges alike most importantly it will allow us also to define 
dependencies between cartridges. Groups should also be flexible enough to not 
only contain single cartridges but also groups, allowing a hierarchical 
Dependencies are defined at the group level, describing the dependency 
relationship of immediate children.
Other group specific properties could be defined which will apply to all 
immediate group members.
The subscription model will be extended to subscribe to a group in addition to 
a cartridge (either or). To generalize we added the concept of a Subscribable 
which can be either a group or cartridge.
Since a Subscribale (or more specifically a group) doesn’t have scaling 
characteristics, it seemed appropriate to add the concept of Scalable, 
describing the scalable nature of a cartridge.
Below is a more detailed description and a corresponding diagram:
“Class diagram” of the logical model showing the existing description files in 
the Diadem model, and the proposed changes.
Description: see attachment Slide1.png
Some notes on the new dependency system:
  * A Group can be subscribed or a Scalable can be subscribed, whereas
    today, a Cartridge is subscribed.
      o The new Scalable entity is needed because currently
        Subscriptions own autoscale policies, but that doesn’t work when
        a Group contains more than one Cartridge, because we want one
        autoscale policy per Cartridge. However we can’t couple the
        autoscale policy to the Cartridge because it may be re-used in
        other Subscriptions where a different autoscale policy is desired
  * children is the set of Subscribables in the Group.
      o This supports hierarchical Groups.
  * collocate says “these Children must be physically next to each other”
  * startupOrder is a Set of asymmetric pairs (A, B) where A points to B
    but B doesn’t point to A.
      o These pairs define a DAG, which we use to perform a topological
        sort of the children and get a /partial/ ordering.
      o Any member of “children” not in a pair is a singleton started in
        parallel to everything else.
  * The killBehaviour says that when node X dies, we must kill:
      o Everything else (for applications where the loss of any member
        of “children” cannot be recovered with less impact
      o Nothing else (for applications where any element can be restarted)
      o Or just the things “upstream” in the startup order (for
        conventional tiered applications)
We think this describes all the use-cases we’ll ever see with the exception of 
things like “n instances of A depends on B”; that can be considered in the 
future as an enhancement. 
Startup use cases considered
  * All start in parallel: don’t specify any pairs => everything is
    equivalent => all start in parallel
  * All start one-after-another (total order): specify a set of pairs
    such that there is a single contiguous list covering all the nodes,
    e.g. A -> B -> C -> D -> E
  * Some start-up dependencies: specify as necessary e.g. [[MiddleTier
    -> Database], [Logging]]
Kill use cases considered
  * One dies => all die: set killBehaviour to KillAll
  * One dies => nothing happens: set killBehaviour to killNone
  * One dies => its dependancies die (aka “restart from here”): specify
    a startupOrder DAG and killBehaviour=StartupOrder. E.g. [[MiddleTier
    -> Database], [Logging]], Database dies => MiddleTier is restarted,
    Logging untouched.
Example instantiation
Description: see attachment Slide2.png
From: Lakmal Warusawithana [] 
Sent: Sunday, March 30, 2014 9:09 PM

Subject: Re: [Discuss] Grouping of services (cartridges)
On Mon, Mar 31, 2014 at 7:47 AM, Martin Eppel (meppel) <> 
I totally agree, I am also not in favor of complicating existing components 
(e.g. autoscaler).  
However, I am not sure what the alternative might be to solve the requirements 
(e.g. item 1) mentioned below.
The suggestion I made to enhance the autoscaler / rules are based on my 
understanding that the autoscaler already handles similar actions (e.g. VM 
startup, scaling and termination).
It seems to me a logical extension to add additional knowledge to handle the 
boot sequencing, dependent scaling and dependent termination to the autoscaler 
defined by optional properties in the autoscaler policy.
>From my current point of view, CAMP seems a fairly complex specification which 
might require quite some changes to adopt. 
Sorry I could not go through CAMP in detail. Should spend sometime. 
I do agree that alternatives might exist and should be discussed !? 
Alternatives ?
+1 for alternatives, we should not take CAMP as it is, we should see how we 
can match with existing Stratos workflow IMO.
From: damitha kumarage [] 
Sent: Sunday, March 30, 2014 7:59 AM

Subject: Re: [Discuss] Grouping of services (cartridges)
Please see my inline comment,
On Sat, Mar 29, 2014 at 11:53 AM, Isuru Haththotuwa <> wrote:
Hi Martin,
On Fri, Mar 28, 2014 at 11:19 PM, Martin Eppel (meppel) <> 
I think this property will be a quite useful piece to solve the grouping 
I also would like to suggest to add the serviceGroup to the topology map (in 
case it is not yet available in the topology map). This  will help to tie 
together cartridges (or services) in the autoscaler and , for example enable 
synchronized auto scaling behavior of services within a service group, like 
synchronized scaling, sequenced boot up, etc …. 
In addition the autoscaler should be enhanced to add additional (but optional 
properties) in the auto scaling policy related to a service group to govern 
the respective auto scaling behavior.
For example, related properties should identify a service group and other 
related properties to define dependencies between the various cartridges in the 
service group like boot sequence, scale up / down ratios, termination 
dependencies, etc … . The property set (or json structure ) should be fairly 
flexible as we are just about to explore this new feature and should be easily 
I would think that these additions will also prove useful to integrate in the 
long term with CAMP (or other spec) but will help to solve more immediate 

Yes, these are very valid points in coming up with a proper grouping 
architecture for services. Thank you for bringing them up.

As I understand, what Sajith has done here is enabling static grouping of 
services, by using a property for that in the cartridge deployment time. What 
we are trying to achieve in long term is dynamic grouping of services, so that 
we can group any available service at runtime seamlessly, according to CAMP 
specification (or some other suitable way). 

I see three things here
1) Grouping and resolve inter-dependancies between  cartridges. 
2) Composite Artifact deployment (May be if required, according to the above 
grouping and inter-dependancy( of cartridges. There can also be intra-
dependancies  inside a cartridge in case of artifact deployment)
3) Improved nonitoring and health check of above intricacies.
Instead of adopt CAMP to solve above, I think it is better to discuss and find 
ways that fits most naturally to the existing Stratos architecture(Unless CAMP 
is widely adopted and we are compelled to adhere).  Is there a way we can 
solve this without doing major changes to existing components? For example 
without complicating autoscaler policies/logic as suggested by Martin above?

Damitha Kumarage

Lakmal Warusawithana
Software Architect; WSO2 Inc.
Mobile : +94714289692
Blog :

Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692
Blog : 


Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692
Blog :

View raw message