stratos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sanjiva Weerawarana <sanj...@wso2.com>
Subject Re: proposed architectural changes to Stratos
Date Thu, 10 Oct 2013 16:40:35 GMT
Looks good!


On Thu, Oct 10, 2013 at 4:54 PM, Lakmal Warusawithana <lakmal@wso2.com>wrote:

> Hi folks,
>
> Based on several feedback, here I re-draw the Stratos Architecture
> diagram. Please share your thoughts.
>
> [image: Inline image 1]
>
>
> On Thu, Oct 10, 2013 at 12:34 PM, Reka Thirunavukkarasu <reka@wso2.com>wrote:
>
>> Hi Lakmal/Devs
>>
>> Would like to be a volunteer for the ADC pub/sub model and the topology
>> handling in the Cloud controller. Will start up with a discussion for them
>> in a separate thread.
>>
>> Thanks,
>> Reka
>>
>>
>> On Thu, Oct 10, 2013 at 9:08 AM, Lakmal Warusawithana <lakmal@wso2.com>wrote:
>>
>>> +1
>>>
>>>
>>>
>>> On Thu, Oct 10, 2013 at 7:36 AM, Pradeep Fernando <pradeepfn@gmail.com>wrote:
>>>
>>>> Hi Lakmal/Devs,
>>>>
>>>> Like the new approach... Looks like the Stratos controller road map
>>>> items are more or less unaffected. I will own the Stratos Manager RESTful
>>>> APIs. (I have already started on this)
>>>>
>>>> Furthermore I will start a discussion on the Stratos new GUI as well.
>>>> Once we all agree, we can initiate the initialization bits in parallel to
>>>> REST API.
>>>>
>>>> Thanks,
>>>> --Pradeep
>>>> On Tue, Oct 8, 2013 at 11:17 AM, Lakmal Warusawithana <lakmal@wso2.com>wrote:
>>>>
>>>>> Hi devs,
>>>>>
>>>>> Shall we start looking into detail of this. Here(below) what I
>>>>> identified individual items for implementing this. we can make a branch
and
>>>>> start working on this (some POC, then alpha ). May be we can do this
as
>>>>> 4.0.0 release, because it has major improvement of the architecture.
>>>>>  Please add/review this items. Volunteers can own these task :)
>>>>>
>>>>> Please share your thoughts.
>>>>>
>>>>> Common:
>>>>>
>>>>>    - Component Topic Pub/Sub
>>>>>    - Health Checker
>>>>>    - Nagios
>>>>>    - define all message structures
>>>>>
>>>>> Cartridge Agent:
>>>>>
>>>>>    - Generic Agent -> better if we can use Python
>>>>>    - pub/sub based DepSync
>>>>>
>>>>> Load Balancer:
>>>>>
>>>>>    - LB Subscribe to topic and create endpoints
>>>>>    - test with multiple LBs
>>>>>    - Test with external LBs (like HA-Proxy)
>>>>>    - LB stat publisher
>>>>>    - Remove Autoscalar from current ELB
>>>>>
>>>>> Cloud Controller :
>>>>>
>>>>>    - Move ADC to CC
>>>>>    - Support pub/sub based ADC
>>>>>    - Event aggregation
>>>>>    - Dynamic Payloads
>>>>>    - Topology handling
>>>>>
>>>>> Stratos Manager
>>>>>
>>>>>    - Interface to define Cartridges
>>>>>    - REST APIs - Json
>>>>>    - New GUI/ without depending carbon framework.
>>>>>    - CLI to work with new REST APIs
>>>>>    - Policy handling
>>>>>
>>>>> AutoScalar :
>>>>>
>>>>>    - New component call autoscalar
>>>>>    - Rule Engine
>>>>>    - Start with simple policy and then go to complex
>>>>>
>>>>> Event Process Engine:
>>>>>
>>>>>    - New component
>>>>>    - May be we can use wso2 cep
>>>>>    - start with simple rule set and go to complex
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Oct 7, 2013 at 10:47 PM, Lakmal Warusawithana <lakmal@wso2.com
>>>>> > wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> As Sanjiva mention, in this architecture will address issues that
has
>>>>>> in current Stratos. With this all major components will loosely coupled,
>>>>>> give more extendability. All communication will used standard pub/sub.
Also
>>>>>> it is very easy to integrate with other external components.
>>>>>>
>>>>>> Please find the first cut of sequence diagram that I come across.
>>>>>>
>>>>>> thanks
>>>>>>
>>>>>>
>>>>>> On Mon, Oct 7, 2013 at 7:33 AM, Sanjiva Weerawarana <sanjiva@wso2.com
>>>>>> > wrote:
>>>>>>
>>>>>>> In a recent F2F meeting with Luca Martini and team from Cisco,
some
>>>>>>> of us discussed several architectural changes that will significantly
>>>>>>> improve Stratos' capabilities and flexibility. The main driving
forces
>>>>>>> behind the proposed changes are:
>>>>>>>
>>>>>>>    - make the overall architecture more resilient and pluggable
by
>>>>>>>    using a message bus as the core communication system
>>>>>>>    - integrate both real-time and rule-based decision making
into
>>>>>>>    the decision making architecture
>>>>>>>    - support both HTTP and non-HTTP traffic in an equal manner
for
>>>>>>>    load balancing and elastic scaling purposes
>>>>>>>    - make the architecture support any LB, including hardware
ones
>>>>>>>
>>>>>>> The following slightly complex picture attempts to explain the
>>>>>>> proposed changes:
>>>>>>>
>>>>>>> [image: Inline image 1]
>>>>>>>
>>>>>>> Let me try to explain a bit more in detail the runtime view as
>>>>>>> that'll show the changes:
>>>>>>>
>>>>>>>    - Traffic comes in thru any load balancer (e.g. HAProxy for
>>>>>>>    non-HTTP traffic and our LB for HTTP traffic). LB routes based
on its
>>>>>>>    configuration which is periodically updated by the Cloud Controller
via
>>>>>>>    topology update messages. LB also fires status events into
the message bus,
>>>>>>>    and possibly directly into the real-time event processor.
>>>>>>>
>>>>>>>    - The event processing engine does temporal (i.e. time based)
>>>>>>>    queries to analyze all the event streams that are being sent
to it and send
>>>>>>>    summarized info to the Autoscaler. Basically the event processing
engine is
>>>>>>>    an event aggregator/accumulator .. it takes lots of events
and produces
>>>>>>>    messages that the autoscaler uses to make elasticity decisions
in a more
>>>>>>>    granular fashion.
>>>>>>>
>>>>>>>    - The Autoscaler runs a rule engine to make auto scaling
>>>>>>>    decisions based on all the messages it has received. (Not
different from
>>>>>>>    now except for the introduction of a rule engine.) The output
of the rule
>>>>>>>    engine runs are a message to the Cloud Controller.
>>>>>>>
>>>>>>>    - The Cloud Controller sends instructions via jClouds to some
>>>>>>>    IaaS to do something. It also listens to messages from instances
and
>>>>>>>    updates the routing topology periodically. Topology updates
fire messages
>>>>>>>    on a different topic that the LBs listen to.
>>>>>>>
>>>>>>>    - Agents in each booted up instance can fire various events
-
>>>>>>>    some to the message bus and some to the realtime event stream.
These events
>>>>>>>    range from "I'm up and ready to go" to "my guy is down" etc..
>>>>>>>
>>>>>>>    - Another component (not shown in picture) can of course be
>>>>>>>    external monitoring systems like Nagios. Those can fire events
into either
>>>>>>>    event stream as well - for example CPU load info can be pumped
into the
>>>>>>>    event processing engine as frequently as possible and let
the event
>>>>>>>    processor analyze over a period of time and alert the autoscaler
if there's
>>>>>>>    something to worry about further.
>>>>>>>
>>>>>>> Overall its not major changes but a bunch of refactoring to make
the
>>>>>>> system more flexible and resilient. It also removes any limits
on the kinds
>>>>>>> of LBs or types of cartridges possible as all communication between
the
>>>>>>> systems is thru asynchronous messaging with no assumptions about
OS/Java
>>>>>>> versions. (The current Hazelcast approach has that limitation.)
>>>>>>>
>>>>>>> Lakmal was going to draw some sequence diagrams to further explain
>>>>>>> the runtime. Lakmal please send when you're ready.
>>>>>>>
>>>>>>> Lets discuss via the list and I look forward a lively f2f discussion
>>>>>>> at the upcoming bar camp in San Francisco.
>>>>>>>
>>>>>>> Sanjiva.
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Lakmal Warusawithana
>>>>>> Software Architect; WSO2 Inc.
>>>>>> Mobile : +94714289692
>>>>>> Blog : http://lakmalsview.blogspot.com/
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Lakmal Warusawithana
>>>>> Software Architect; WSO2 Inc.
>>>>> Mobile : +94714289692
>>>>> Blog : http://lakmalsview.blogspot.com/
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Pradeep Fernando.
>>>> http://pradeepfernando.blogspot.com/
>>>>
>>>
>>>
>>>
>>> --
>>> Lakmal Warusawithana
>>> Software Architect; WSO2 Inc.
>>> Mobile : +94714289692
>>> Blog : http://lakmalsview.blogspot.com/
>>>
>>>
>>
>>
>> --
>> Reka Thirunavukkarasu
>> Software Engineer,
>> WSO2, Inc.:http://wso2.com,
>> Mobile: +94776442007
>>
>>
>>
>
>
> --
> Lakmal Warusawithana
> Software Architect; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>


-- 
Sanjiva Weerawarana, Ph.D.
Founder, Chairman & CEO; WSO2, Inc.;  http://wso2.com/
email: sanjiva@wso2.com; phone: +94 11 763 9614; cell: +94 77 787 6880 | +1
650 265 8311
blog: http://sanjiva.weerawarana.org/

Lean . Enterprise . Middleware

Mime
View raw message