stratos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Reka Thirunavukkarasu <r...@wso2.com>
Subject Re: proposed architectural changes to Stratos
Date Thu, 10 Oct 2013 07:04:14 GMT
Hi Lakmal/Devs

Would like to be a volunteer for the ADC pub/sub model and the topology
handling in the Cloud controller. Will start up with a discussion for them
in a separate thread.

Thanks,
Reka


On Thu, Oct 10, 2013 at 9:08 AM, Lakmal Warusawithana <lakmal@wso2.com>wrote:

> +1
>
>
>
> On Thu, Oct 10, 2013 at 7:36 AM, Pradeep Fernando <pradeepfn@gmail.com>wrote:
>
>> Hi Lakmal/Devs,
>>
>> Like the new approach... Looks like the Stratos controller road map items
>> are more or less unaffected. I will own the Stratos Manager RESTful APIs.
>> (I have already started on this)
>>
>> Furthermore I will start a discussion on the Stratos new GUI as well.
>> Once we all agree, we can initiate the initialization bits in parallel to
>> REST API.
>>
>> Thanks,
>> --Pradeep
>> On Tue, Oct 8, 2013 at 11:17 AM, Lakmal Warusawithana <lakmal@wso2.com>wrote:
>>
>>> Hi devs,
>>>
>>> Shall we start looking into detail of this. Here(below) what I
>>> identified individual items for implementing this. we can make a branch and
>>> start working on this (some POC, then alpha ). May be we can do this as
>>> 4.0.0 release, because it has major improvement of the architecture.
>>>  Please add/review this items. Volunteers can own these task :)
>>>
>>> Please share your thoughts.
>>>
>>> Common:
>>>
>>>    - Component Topic Pub/Sub
>>>    - Health Checker
>>>    - Nagios
>>>    - define all message structures
>>>
>>> Cartridge Agent:
>>>
>>>    - Generic Agent -> better if we can use Python
>>>    - pub/sub based DepSync
>>>
>>> Load Balancer:
>>>
>>>    - LB Subscribe to topic and create endpoints
>>>    - test with multiple LBs
>>>    - Test with external LBs (like HA-Proxy)
>>>    - LB stat publisher
>>>    - Remove Autoscalar from current ELB
>>>
>>> Cloud Controller :
>>>
>>>    - Move ADC to CC
>>>    - Support pub/sub based ADC
>>>    - Event aggregation
>>>    - Dynamic Payloads
>>>    - Topology handling
>>>
>>> Stratos Manager
>>>
>>>    - Interface to define Cartridges
>>>    - REST APIs - Json
>>>    - New GUI/ without depending carbon framework.
>>>    - CLI to work with new REST APIs
>>>    - Policy handling
>>>
>>> AutoScalar :
>>>
>>>    - New component call autoscalar
>>>    - Rule Engine
>>>    - Start with simple policy and then go to complex
>>>
>>> Event Process Engine:
>>>
>>>    - New component
>>>    - May be we can use wso2 cep
>>>    - start with simple rule set and go to complex
>>>
>>>
>>>
>>>
>>> On Mon, Oct 7, 2013 at 10:47 PM, Lakmal Warusawithana <lakmal@wso2.com>wrote:
>>>
>>>> Hi,
>>>>
>>>> As Sanjiva mention, in this architecture will address issues that has
>>>> in current Stratos. With this all major components will loosely coupled,
>>>> give more extendability. All communication will used standard pub/sub. Also
>>>> it is very easy to integrate with other external components.
>>>>
>>>> Please find the first cut of sequence diagram that I come across.
>>>>
>>>> thanks
>>>>
>>>>
>>>> On Mon, Oct 7, 2013 at 7:33 AM, Sanjiva Weerawarana <sanjiva@wso2.com>wrote:
>>>>
>>>>> In a recent F2F meeting with Luca Martini and team from Cisco, some of
>>>>> us discussed several architectural changes that will significantly improve
>>>>> Stratos' capabilities and flexibility. The main driving forces behind
the
>>>>> proposed changes are:
>>>>>
>>>>>    - make the overall architecture more resilient and pluggable by
>>>>>    using a message bus as the core communication system
>>>>>    - integrate both real-time and rule-based decision making into the
>>>>>    decision making architecture
>>>>>    - support both HTTP and non-HTTP traffic in an equal manner for
>>>>>    load balancing and elastic scaling purposes
>>>>>    - make the architecture support any LB, including hardware ones
>>>>>
>>>>> The following slightly complex picture attempts to explain the
>>>>> proposed changes:
>>>>>
>>>>> [image: Inline image 1]
>>>>>
>>>>> Let me try to explain a bit more in detail the runtime view as that'll
>>>>> show the changes:
>>>>>
>>>>>    - Traffic comes in thru any load balancer (e.g. HAProxy for
>>>>>    non-HTTP traffic and our LB for HTTP traffic). LB routes based on
its
>>>>>    configuration which is periodically updated by the Cloud Controller
via
>>>>>    topology update messages. LB also fires status events into the message
bus,
>>>>>    and possibly directly into the real-time event processor.
>>>>>
>>>>>    - The event processing engine does temporal (i.e. time based)
>>>>>    queries to analyze all the event streams that are being sent to it
and send
>>>>>    summarized info to the Autoscaler. Basically the event processing
engine is
>>>>>    an event aggregator/accumulator .. it takes lots of events and produces
>>>>>    messages that the autoscaler uses to make elasticity decisions in
a more
>>>>>    granular fashion.
>>>>>
>>>>>    - The Autoscaler runs a rule engine to make auto scaling decisions
>>>>>    based on all the messages it has received. (Not different from now
except
>>>>>    for the introduction of a rule engine.) The output of the rule engine
runs
>>>>>    are a message to the Cloud Controller.
>>>>>
>>>>>    - The Cloud Controller sends instructions via jClouds to some IaaS
>>>>>    to do something. It also listens to messages from instances and updates
the
>>>>>    routing topology periodically. Topology updates fire messages on a
>>>>>    different topic that the LBs listen to.
>>>>>
>>>>>    - Agents in each booted up instance can fire various events - some
>>>>>    to the message bus and some to the realtime event stream. These events
>>>>>    range from "I'm up and ready to go" to "my guy is down" etc..
>>>>>
>>>>>    - Another component (not shown in picture) can of course be
>>>>>    external monitoring systems like Nagios. Those can fire events into
either
>>>>>    event stream as well - for example CPU load info can be pumped into
the
>>>>>    event processing engine as frequently as possible and let the event
>>>>>    processor analyze over a period of time and alert the autoscaler if
there's
>>>>>    something to worry about further.
>>>>>
>>>>> Overall its not major changes but a bunch of refactoring to make the
>>>>> system more flexible and resilient. It also removes any limits on the
kinds
>>>>> of LBs or types of cartridges possible as all communication between the
>>>>> systems is thru asynchronous messaging with no assumptions about OS/Java
>>>>> versions. (The current Hazelcast approach has that limitation.)
>>>>>
>>>>> Lakmal was going to draw some sequence diagrams to further explain the
>>>>> runtime. Lakmal please send when you're ready.
>>>>>
>>>>> Lets discuss via the list and I look forward a lively f2f discussion
>>>>> at the upcoming bar camp in San Francisco.
>>>>>
>>>>> Sanjiva.
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Lakmal Warusawithana
>>>> Software Architect; WSO2 Inc.
>>>> Mobile : +94714289692
>>>> Blog : http://lakmalsview.blogspot.com/
>>>>
>>>>
>>>
>>>
>>> --
>>> Lakmal Warusawithana
>>> Software Architect; WSO2 Inc.
>>> Mobile : +94714289692
>>> Blog : http://lakmalsview.blogspot.com/
>>>
>>>
>>
>>
>> --
>> Pradeep Fernando.
>> http://pradeepfernando.blogspot.com/
>>
>
>
>
> --
> Lakmal Warusawithana
> Software Architect; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>


-- 
Reka Thirunavukkarasu
Software Engineer,
WSO2, Inc.:http://wso2.com,
Mobile: +94776442007

Mime
View raw message