stratos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Martin Eppel (meppel)" <mep...@cisco.com>
Subject RE: [Discuss] Grouping of services (cartridges)
Date Sat, 17 May 2014 21:44:23 GMT
Absolutely,

We want to incorporate the feedback of all the experts but in the meantime we can get started
based on what we know and then gradually enhance.

IMHO it would also make sense to start developing the feature on a separate branch (of RC3
tag ) and gradually improve the feature until we feel it is ready to be incorporated into
the mainline, this would allow us to have something ready by the end of the month without
destabilizing the master branch ?

Thanks

Martin

From: isuruh@wso2.com [mailto:isuruh@wso2.com] On Behalf Of Isuru Haththotuwa
Sent: Saturday, May 17, 2014 10:16 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org; Melan Jayasingha; Lakmal Warusawithana; Shaheedur Haque
(shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)

Hi Martin,
Thanks a lot for your efforts. Will try to have a look at the code before the call.
IMHO, before adding our changes, it is better to share the intended design with the dev list
and get feedback as well. We might need to come with some sequence diagrams as well. I was
planning to do that after we have the call tomorrow. This is a crucial and a core change,
plus a very useful feature to Stratos, so better to get feedback from the experts in the dev
list. WDYT?
I prepared some JSON definition referring your ones, in which I have only one single JSON
definition for the group subscription. My intention is to re-use the Subscription model that
we have in Stratos Manager and extend it to support groups and dependencies. Lets discuss
during the call, and decide how to proceed.

On Sat, May 17, 2014 at 10:09 PM, Martin Eppel (meppel) <meppel@cisco.com<mailto:meppel@cisco.com>>
wrote:
Hi Isuru, Melan

Attached is the code I started implementing:

1.       Added REST API for application deployment

2.       Added ApplicationCreatedEvent (+ some corresponding messaging infrastructure)

I also attached the sample JSON I used to test the REST API:

Some of the code is experimental, e.g publishing the  ApplicationCreatedEvent  from ServiceUtil
with the purpose to test the messaging infrastructure and to be able to implement and test
the backend changes (application json) in the autoscaler (not there yet).

I think there is still a bunch of things missing like an application deployment / subscriptions
manager (similar to deploying a cartridge, etc …), but I am not entirely sure what you guys
have in mind there.
I do have some idea what should be done in the autoscaler to add dependencies check, so I
was thinking you could focus on the  deployment logic, messaging, etc …  while I take a
look at the autoscaler stuff.
Anyway, this is just a suggestion, take a look and we can discuss more details in our call.

Regards


Martin

From: Martin Eppel (meppel)
Sent: Friday, May 16, 2014 8:16 PM
To: Isuru Haththotuwa

Cc: dev@stratos.incubator.apache.org<mailto:dev@stratos.incubator.apache.org>; Lakmal
Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: RE: [Discuss] Grouping of services (cartridges)

Hi Isuru,

Just confirming the  meeting time Saturday 8.30 PST is ok with me (My time would be Sunday
8.30 AM IST)

Regards

Martin

From: Isuru Haththotuwa [mailto:isuruh@wso2.com<mailto:isuruh@wso2.com>]
Sent: Friday, May 16, 2014 10:50 AM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<mailto:dev@stratos.incubator.apache.org>; Lakmal
Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha
Subject: Re: [Discuss] Grouping of services (cartridges)

Hi Martin,

On Fri, May 16, 2014 at 9:32 PM, Martin Eppel (meppel) <meppel@cisco.com<mailto:meppel@cisco.com>>
wrote:
Hi Isuru,

What about tonight 8:00 pm PST (should be 8:30 am your time Saturday), if this doesn’t work
we can meet up Sun 8:00 pm PST (8:30 am IST ?)
Great! Saturday 8.30 PST is ok with me (My time would be Sunday 8.30 AM IST)

How will we connect, I haven’t used Google hangout yet ?
We can connect through either Google hangout/ Skype, whatever easier for you.

I noticed that emails are getting very slowly processed on dev@stratos list so I am sending
it also directly to you as well,
I too noticed that your last mail got delayed by approx. one day.

Btw, we are trying to get this feature ready (at least in some basic form) by the end of the
month
I'm positive we can get an initial version working by the end of this month, if all goes well.

Thanks

Martin



From: isuruh@wso2.com<mailto:isuruh@wso2.com> [mailto:isuruh@wso2.com<mailto:isuruh@wso2.com>]
On Behalf Of Isuru Haththotuwa
Sent: Thursday, May 15, 2014 9:10 PM
To: dev@stratos.incubator.apache.org<mailto:dev@stratos.incubator.apache.org>
Cc: Lakmal Warusawithana; Shaheedur Haque (shahhaqu); Melan Jayasingha

Subject: Re: [Discuss] Grouping of services (cartridges)

Hi Martin,
I'm in IST. Please suggest a convenient time for you. I would like to clarify several things
on the proposal and the concepts you brought in, simplify it further if possible, and decide
how to proceed.

On Thu, May 15, 2014 at 3:52 AM, Martin Eppel (meppel) <meppel@cisco.com<mailto:meppel@cisco.com>>
wrote:
Hi Isuru, see inline

From: isuruh@wso2.com<mailto:isuruh@wso2.com> [mailto:isuruh@wso2.com<mailto:isuruh@wso2.com>]
On Behalf Of Isuru Haththotuwa
Sent: Wednesday, May 14, 2014 1:12 PM
To: Martin Eppel (meppel)
Cc: dev@stratos.incubator.apache.org<mailto:dev@stratos.incubator.apache.org>; Lakmal
Warusawithana; Shaheedur Haque (shahhaqu)

Subject: Re: [Discuss] Grouping of services (cartridges)

Hi Martin,

On Wed, May 14, 2014 at 2:34 AM, Martin Eppel (meppel) <meppel@cisco.com<mailto:meppel@cisco.com>>
wrote:
Hi Isuru, Shaheed


Ad “kill behaviour”:
+1 for the kill-behavior terms: “kill-dependents”, “kill-all”, “kill-none” (btw,
was left out accidentally, thanks for pointing it out),

Ad single definition:

Makes sense too, however  we should still consider that the json definition can become fairly
complex when multiple components and sub components are defined, that’s the reason I think
it would still make sense to keep it modular, at least as an option.
+1
Martin: +1
(btw, I assume you suggest instead of defining each component in separate files we aggregate
them in the same application  json document, but not like a “in-line” definition of each
component?)
Exactly . This is fine as the first step IMHO.

Martin: +1 ( inline wouldn’t be a good choice IMHO)
Does this include other definitions like cartridge info definition, deployment policy, autoscaling
definition, etc ... ?
No. I agree that if we do this, it will be an unmanageably large JSON definition. We can refer
to the relevant policies (which are already deployed) in the service group definition. The
service group should contain the group specific details IMHO, as the dependencies, startup
orders, kill behaviours, etc. WDYT?

Martin: +1 (see my example json definitions)

We could provide eventually both options, in-line with all components defined in one document
(simple configurations) or modular, where components are deployed (un-deployed) individually
(for large, complex configurations).

In the shared document I provided a full example of the json definition of an application,
and just adding a few components seemed to make the document fairly large or complex (see
chapter “Json example”).
Great! Thanks Martin. Lets do a hangout, preferably on Friday so that we can decide on the
next steps.

Martin: sounds good, Friday would be Thursday my time, which time zone are you in ?


Thanks

Martin

https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit#heading=h.y52cwrsrj31p


From: Isuru Haththotuwa [mailto:isuruh@wso2.com<mailto:isuruh@wso2.com>]
Sent: Tuesday, May 13, 2014 6:08 AM
To: dev@stratos.incubator.apache.org<mailto:dev@stratos.incubator.apache.org>
Cc: Lakmal Warusawithana

Subject: Re: [Discuss] Grouping of services (cartridges)

Hi Martin, Shaheedur and all,
Went through the shared document for the proposal. good work! One small suggestion, additionally
to deploying component JSON definitions one by one, shall we allow to deploy a one single
definition (that will have all the dependencies and other information defined) to be deployed?
WDYT? IMHO this will be more user friendly.


On Wed, May 7, 2014 at 3:28 PM, Shaheedur Haque (shahhaqu) <shahhaqu@cisco.com<mailto:shahhaqu@cisco.com>>
wrote:
A couple of points on “kill behaviour”:


•         I don’t think the terms “asymmetric” and “symmetric” capture the behaviours
I had in mind. For me there were two use cases:

o   Assuming A depends on B and B depends on C

o   And B fails

o   Use case #1 is that only A should be killed and allowed to restart. We could call this
kill behaviour “kill-dependents” since A is dependent on B. This fits the standard n-tier
architecture where A is the front end, B is the application logic and C is the database.

o   Use case #2 is that both A and C should be killed and allowed to restart. We could call
this “kill-all”. This fits the use case of a poorly behaved application where everything
needs a restart on any failure.

•         Did we accidentally drop the third value, i.e. “kill-none”, or did we somehow
convince ourselves this was not needed? I’m actually tending towards the latter since if
kill-none were a usable value, it seems to contradict that there was a dependency in the first
place.
+1.

Thanks, Shaheed

From: Lakmal Warusawithana [mailto:lakmal@wso2.com<mailto:lakmal@wso2.com>]
Sent: 07 May 2014 06:48

To: dev@stratos.incubator.apache.org<mailto:dev@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)



On Tue, May 6, 2014 at 10:33 PM, Martin Eppel (meppel) <meppel@cisco.com<mailto:meppel@cisco.com>>
wrote:
Hi Lakmal,

See inline "Martin:"

From: Lakmal Warusawithana [mailto:lakmal@wso2.com<mailto:lakmal@wso2.com>]
Sent: Tuesday, May 06, 2014 3:25 AM
To: dev@stratos.incubator.apache.org<mailto:dev@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)

Hi Martin,
Thanks for adding content, I had offline discussion with Imesh, Lahiru and IsuruH on this,
we can start implementing step by step. will do this by milestone by milestone approach.

Further I think we can used same topology topic to create this model. With this root node
of the topology will be application. From the auto scaler we can implement application monitor
to handle this. service clusters will be sub set of application.
Martin: +1, dependencies have to be included

As the first step we reused (partitions, auto scaling policy, deployment policy, cartridge
definitions) existing definitions to define application with new requirement like grouping,
dependency, start order ..etc. Shall we create sample json ( with sample auto scaling policies,
deployment policies, cartridges ..etc) to get some idea for the complexity? (still I am figuring
out some definition like subscribable and some real sample json)
Martin: +1, Actually, the json example I posted on Google docs assumes that we'll be reusing
all the existing artifacts (like cartridge definition etc ...). Cartridges, policies, componenents
are referenced by name so each of these have to be deployed separately (as they are today)
and when the application is deployed it will reference the deployed artifacts - what do you
think ?
We'll might have to add some extra properties - I'll work on a complete example and post the
progress on Google Doc.


Then we should divide each work item and start implementing. Ideally in the milestone one
should not break existing, but reused them. Then later we can identifying what need to improve/change
existing components.
Martin: +1, would it speed up / simply implementation if we start working on a branch ?  Also,
I totally agree, all code change should be complementary without breaking any existing features

Later this week we can have google hangout session to discuss further on this.
Martin: I will be ooo / unavailable Thursday (your time) - Tuesday, we can do a hangout either
today,  tomorrow or then again next week Wednesday.

Seems like we have to wait till next Friday, next week, Wednesday and  Thursday Holiday in
Sri Lanka. Anyway will start working on this and will identify initial task and will post
here.

Lets have a hangout at a convenient time to discuss the next steps.

On Mon, May 5, 2014 at 10:24 AM, Martin Eppel (meppel) <meppel@cisco.com<mailto:meppel@cisco.com>>
wrote:
Works for me, I’ll add the content

Thanks

Martin

From: Lakmal Warusawithana [mailto:lakmal@wso2.com<mailto:lakmal@wso2.com>]
Sent: Sunday, May 04, 2014 8:56 PM

To: dev@stratos.incubator.apache.org<mailto:dev@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)

Hi Martin,

Thanks for detail description.  We can create wiki space , but IMHO its easy if we can use
public google doc. I have created [1] shall we add our proposal into it. May be we need some
hangout session to discuss further on this.

[1]https://docs.google.com/document/d/1qnArxF8obhbjMLUbPAENQMn94fPSd3AVqO41mxhnJXE/edit


On Mon, May 5, 2014 at 8:29 AM, Martin Eppel (meppel) <meppel@cisco.com<mailto:meppel@cisco.com>>
wrote:
Hi Lakmal,

I summarized and formatted the discussion points from the previous email(s) thread for better
readability and reference.
Btw, what is the appropriate way going forward to elevate the discussion to a feature request
(JIRA) ? Also, how do we typically discuss feature requests like this, the email format makes
it somewhat difficult to read and it is easy to overlook posted discussion points ? Is there
a wiki to post and discuss proposals like this ?

Thanks

Martin

Grouping of Cartridges / Services:

1. Application model
   1.1 General model
       See proposal below
   1.2 Json
       see proposal below

2. Application deployment
   2.1 Deployment
       + application deployment (deploy)
       + application removal (undeploy)
   2.2 Subscription
       + subscription (subscribe)
       + cancelation of subscription (unsubscribe)

2. Distributing Application deployment
   2.1 Publishing application deployment
       + new topic to publish application events, map, status
   2.2 Subscribing to published application deployment
       + Autoscaler

3. Dependency calculation
   3.1 Building static dependency tree
       3.1.1 Calculating dependency tree, dependency check in Autoscaler
           + Dependency tree determines all dependencies,
           + symmetrical / asymmetrical dependencies
           + Dependency checks member status for all dependencies
             successful for at least one member in ACTIVE state, failed for no member in ACTIVE
state
           3.1.1.1 Enhancing autoscaler rules
                   + Integrate with
                     mincheck.drl to control instance creation, termination rule to control
instance termination


4. Application event model
   + provides state information for application, application components like
     Up, Down, partial up



From: Martin Eppel (meppel)
Sent: Thursday, May 01, 2014 3:26 PM

To: dev@stratos.incubator.apache.org<mailto:dev@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: RE: [Discuss] Grouping of services (cartridges)

Hi Lakmal at al

Here is a first draft for an application model as a starting point, it builds on the previous
discussions (and probably leaves out a few things):
Also, to add to Shaheeds point there is probably an event model around this as well.

Definitions:
+ composite structure
+ A component is sectioned into description, components, configuration, dependencies.
+ Components are typed (application, subscribables, scalable)
+ Application is also component

+ description     ...   properties (like name, type)
+ components      ...   children, nested
+ configuration   ...   properties
+ dependencies    ...   puts immediate components (children) in relation

Generic Component format:
   + description (component specific properties)
   + components (by reference)
   + configuration
      - described through properties
          + properties can be grouped, group can be referenced
   + dependencies
      - described through properties


In json:

Generic Component:

Component:

{
      "description": [
      ],
      "configuration":[
      ],
      "components":[
      ],
      "dependencies":[
      ]
}


Application:

{
    "description": [
            {"name":"application name"},
            {"types":["application", "subscribable"]}
      ],
      "configuration":[
            {"deployment":"deployment_policy_name"}
      ],
      "components":[
            {"subscribables": ["subscribable name",
                                       "subscribable name",
                                       "subscribable name"]}
      ],
      "dependencies":[
            {"startup_order": [
            {"dependant name": "subscribable name"},
                  {"dependant name": "subscribable name"}
            ]},
            {"kill_behavior": "symmetrical or asymmetrical"}
      ]
}

Subscribable (aka groups):

{
    "description": [
            {"name":"component name"},
            {"types":["subscribable"]}
      ],
      "configuration":[
      ],
      "components":[
            {"subscribables": ["subscribable name",
                              "subscribable name",
                              "subscribable name"]}
      ],
      "dependencies":[
            {"startup_order": [
            {"dependant name": "subscribable name"},
                  {"dependant name": "subscribable name"}
            ]},
            {"kill_behavior": "symmetrical / asymmetrical"}
      ]
}

Cartridge Component:
{
    "description": [
            {"name":"component name"},
            {"types":["subscribable", "scalable"]}
      ],
      "configuration":[
            {"scaling":"autoscaling policy name"},
            {"info":"cartridge info"}
      ],
      "components":[
      ],
      "dependencies":[
      ]
}


Thanks

Martin

From: Lakmal Warusawithana [mailto:lakmal@wso2.com<mailto:lakmal@wso2.com>]
Sent: Thursday, May 01, 2014 1:18 AM
To: Shaheedur Haque (shahhaqu)
Cc: dev@stratos.incubator.apache.org<mailto:dev@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)



On Thu, May 1, 2014 at 1:43 PM, Shaheed Haque <shahhaqu@cisco.com<mailto:shahhaqu@cisco.com>>
wrote:
Hi,

Sorry for top-posting, but one additional thing that I feel should be considered is a minimal
set of lifecycle events for the group; for example:

- "all members of a group are now active", aka "group active".

- "at least one member of an existing group has failed", aka "group down".

This is clearly an add-on to the core functionality but is needed to round the feature out.


+1

Thanks, Shaheed


On Thursday 01 May 2014 07:46:46 Lakmal Warusawithana wrote:
Hi Martin,

With the current implementation I think and agree its better to go with json as defining configuration
data. Summarizing proposal and the discussion I can think following data model. Here I am
thinking implementing single configuration data (application deployment definition) model,
which can define full configuration of an composite application. we should take it down to
several milestones. we can start on enhancing backend services in initial milestone and later
will come with aggregating them. All of your thoughts welcome.

Can you come up with some definition format. (may be json format). Also see my inline comment.

On Wed, Apr 30, 2014 at 11:03 PM, Martin Eppel (meppel) <meppel@cisco.com<mailto:meppel@cisco.com>>
wrote:
Hi Lakmal,

Below are ideas on how to break up the feature into a high level ToDo list (with some suggestions
and questions) :

+ Defining configuration data format – for example json or chef / recipe
   ++ would using recipes affect all configuration data or is it restricted to the grouping
/ dependency configuration only ?

IMHO, its may easy with json, considering current existing services.


+ Adding the new model to stratos manager and integrate with existing model:
   ++ Group model (nested groups, group properties)
    ++ Dependency model
    ++ in the current model we have a Subscriptions and Deployment manager, I think we need
something similar to handle the groups which in turn invokes deployment / subscriptions manager


+1

+ Defining runtime dependency conditions:
      ++ Calculating the dependency for a cartridge (service) dynamically by walking the dependency
tree and checking the state of dependencies (e.g. member state)


+1

+ Integrate the dynamic aspect of the dependency model (checking the conditions of dependencies
and act accordingly):
      ++ As of now I think it has to be integrated with the autoscaler as it the entity which
controls instances (members) and checks their states

Yes, we need to enhance auto scaler

      ++ How is dependency retrieved from configuration data (e.g. published through topology,
beans, etc ….) ?

We need to think on more this. IMO, we may need another topic (application deployment definition)
to retrieved the config data. SM can define this based on provide definition, auto scaler
and who ever need can get that. Topology is maintaining current running state (currently its
for a service, we need to enhance it to application), so we can used "application deployment
definition" data to check with current state.

      ++ Integrate dependency checks into the runtime portion of the autoscaler, e.g.  add
the checks to the monitor functionality of a ClusterMonitor with respective actions (spawning
 / terminating instances)


I think we can coupled auto scaling policy with individual cartridges/services.

What do you think ?


We may need to think about deployment policy, I think with the application deployment definition
(with some coupling ,like some cartridges may need to spin up same partition..etc). May be
we are redefining deployment policy with application deployment definition. (we may obsolete
deployment policy)

Thanks

Martin

From: Lakmal Warusawithana [mailto:lakmal@wso2.com<mailto:lakmal@wso2.com>]
Sent: Wednesday, April 30, 2014 4:30 AM

To: dev@stratos.incubator.apache.org<mailto:dev@stratos.incubator.apache.org>
Cc: Shaheedur Haque (shahhaqu)
Subject: Re: [Discuss] Grouping of services (cartridges)

Hi Martin,

These use cases are very valid, and we should integrate them into 4.1.0. Will go through in
detail and see how we can incorporate into Stratos.
I have one question, why do we need hierarchical grouping, (I mean group inside a group) and
use case? Can't we have flat?

On Wed, Apr 30, 2014 at 9:56 AM, Martin Eppel (meppel) <meppel@cisco.com<mailto:meppel@cisco.com>>
wrote:
Hi Lakmal

Below is a further enhanced proposal to add grouping to apache stratos:

In a nutshell:

By grouping cartridges together we can define characteristics which apply to all cartridges
alike most importantly it will allow us also to define dependencies between cartridges. Groups
should also be flexible enough to not only contain single cartridges but also groups, allowing
a hierarchical structure.

Dependencies are defined at the group level, describing the dependency relationship of immediate
children.
Other group specific properties could be defined which will apply to all immediate group members.

The subscription model will be extended to subscribe to a group in addition to a cartridge
(either or). To generalize we added the concept of a Subscribable which can be either a group
or cartridge.

Since a Subscribale (or more specifically a group) doesn’t have scaling characteristics,
it seemed appropriate to add the concept of Scalable, describing the scalable nature of a
cartridge.

Below is a more detailed description and a corresponding diagram:

“Class diagram” of the logical model showing the existing description files in the Diadem
model, and the proposed changes.

Description: see attachment Slide1.png

Some notes on the new dependency system:

  * A Group can be subscribed or a Scalable can be subscribed, whereas
    today, a Cartridge is subscribed.
      o The new Scalable entity is needed because currently
        Subscriptions own autoscale policies, but that doesn’t work when
        a Group contains more than one Cartridge, because we want one
        autoscale policy per Cartridge. However we can’t couple the
        autoscale policy to the Cartridge because it may be re-used in
        other Subscriptions where a different autoscale policy is desired
  * children is the set of Subscribables in the Group.
      o This supports hierarchical Groups.
  * collocate says “these Children must be physically next to each other”
  * startupOrder is a Set of asymmetric pairs (A, B) where A points to B
    but B doesn’t point to A.
      o These pairs define a DAG, which we use to perform a topological
        sort of the children and get a /partial/ ordering.
      o Any member of “children” not in a pair is a singleton started in
        parallel to everything else.
  * The killBehaviour says that when node X dies, we must kill:
      o Everything else (for applications where the loss of any member
        of “children” cannot be recovered with less impact
      o Nothing else (for applications where any element can be restarted)
      o Or just the things “upstream” in the startup order (for
        conventional tiered applications)

We think this describes all the use-cases we’ll ever see with the exception of things like
“n instances of A depends on B”; that can be considered in the future as an enhancement.

Startup use cases considered

  * All start in parallel: don’t specify any pairs => everything is
    equivalent => all start in parallel
  * All start one-after-another (total order): specify a set of pairs
    such that there is a single contiguous list covering all the nodes,
    e.g. A -> B -> C -> D -> E
  * Some start-up dependencies: specify as necessary e.g. [[MiddleTier
    -> Database], [Logging]]

Kill use cases considered

  * One dies => all die: set killBehaviour to KillAll
  * One dies => nothing happens: set killBehaviour to killNone
  * One dies => its dependancies die (aka “restart from here”): specify
    a startupOrder DAG and killBehaviour=StartupOrder. E.g. [[MiddleTier
    -> Database], [Logging]], Database dies => MiddleTier is restarted,
    Logging untouched.

Example instantiation

Description: see attachment Slide2.png

Thanks

Martin


From: Lakmal Warusawithana [mailto:lakmal@wso2.com<mailto:lakmal@wso2.com>]
Sent: Sunday, March 30, 2014 9:09 PM

To: dev@stratos.incubator.apache.org<mailto:dev@stratos.incubator.apache.org>
Subject: Re: [Discuss] Grouping of services (cartridges)



On Mon, Mar 31, 2014 at 7:47 AM, Martin Eppel (meppel) <meppel@cisco.com<mailto:meppel@cisco.com>>
wrote:
I totally agree, I am also not in favor of complicating existing components (e.g. autoscaler).

However, I am not sure what the alternative might be to solve the requirements (e.g. item
1) mentioned below.
The suggestion I made to enhance the autoscaler / rules are based on my understanding that
the autoscaler already handles similar actions (e.g. VM startup, scaling and termination).
It seems to me a logical extension to add additional knowledge to handle the boot sequencing,
dependent scaling and dependent termination to the autoscaler defined by optional properties
in the autoscaler policy.

From my current point of view, CAMP seems a fairly complex specification which might require
quite some changes to adopt.

Sorry I could not go through CAMP in detail. Should spend sometime.


I do agree that alternatives might exist and should be discussed !? Alternatives ?

+1 for alternatives, we should not take CAMP as it is, we should see how we can match with
existing Stratos workflow IMO.


Thanks

Martin

From: damitha kumarage [mailto:damitha23@gmail.com<mailto:damitha23@gmail.com>]
Sent: Sunday, March 30, 2014 7:59 AM
To: dev@stratos.incubator.apache.org<mailto:dev@stratos.incubator.apache.org>

Subject: Re: [Discuss] Grouping of services (cartridges)

Please see my inline comment,

On Sat, Mar 29, 2014 at 11:53 AM, Isuru Haththotuwa <isuruh@wso2.com<mailto:isuruh@wso2.com>>
wrote:
Hi Martin,
On Fri, Mar 28, 2014 at 11:19 PM, Martin Eppel (meppel) <meppel@cisco.com<mailto:meppel@cisco.com>>
wrote:
Hi,

I think this property will be a quite useful piece to solve the grouping issue:

I also would like to suggest to add the serviceGroup to the topology map (in case it is not
yet available in the topology map). This  will help to tie together cartridges (or services)
in the autoscaler and , for example enable synchronized auto scaling behavior of services
within a service group, like synchronized scaling, sequenced boot up, etc ….

In addition the autoscaler should be enhanced to add additional (but optional properties)
in the auto scaling policy related to a service group to govern the respective auto scaling
behavior.

For example, related properties should identify a service group and other related properties
to define dependencies between the various cartridges in the service group like boot sequence,
scale up / down ratios, termination dependencies, etc … . The property set (or json structure
) should be fairly flexible as we are just about to explore this new feature and should be
easily expandable.

I would think that these additions will also prove useful to integrate in the long term with
CAMP (or other spec) but will help to solve more immediate requirements


Yes, these are very valid points in coming up with a proper grouping architecture for services.
Thank you for bringing them up.

As I understand, what Sajith has done here is enabling static grouping of services, by using
a property for that in the cartridge deployment time. What we are trying to achieve in long
term is dynamic grouping of services, so that we can group any available service at runtime
seamlessly, according to CAMP specification (or some other suitable way).

I see three things here
1) Grouping and resolve inter-dependancies between  cartridges.
2) Composite Artifact deployment (May be if required, according to the above grouping and
inter-dependancy( of cartridges. There can also be intra-dependancies  inside a cartridge
in case of artifact deployment)
3) Improved nonitoring and health check of above intricacies.
Instead of adopt CAMP to solve above, I think it is better to discuss and find ways that fits
most naturally to the existing Stratos architecture(Unless CAMP is widely adopted and we are
compelled to adhere).  Is there a way we can solve this without doing major changes to existing
components? For example without complicating autoscaler policies/logic as suggested by Martin
above?
Damitha




--
__________________________________________________________________
Damitha Kumarage
http://people.apache.org/
__________________________________________________________________


--
Lakmal Warusawithana
Software Architect; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/


--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/


--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/





--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/




--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/




--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/



--
Lakmal Warusawithana
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692<tel:%2B94714289692>
Blog : http://lakmalsview.blogspot.com/



--
Thanks and Regards,

Isuru H.
+94 716 358 048<tel:%2B94%20716%20358%20048>





--
Thanks and Regards,

Isuru H.
+94 716 358 048<tel:%2B94%20716%20358%20048>

Mime
View raw message