incubator-cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kelven Yang <kelven.y...@citrix.com>
Subject Re: [Discuss] CloudStack architecture to a loosely-coupled component oriented distributed architecture
Date Mon, 20 Aug 2012 17:53:30 GMT
Murali,

The key point is the shifting to event-driven programming model itself at
orchestration layer, the underlying facility is currently not at critical
path, we will need to do a messaging arbitration here so that we can
plugin a suitable MQ selection in a fairly easy step. One thing I'm sure
is that we won't use two separated MQ solutions in one system.

We also need to think twice of tight-coupled listener pattern across
components, if listeners from different components are implicitly required
to participate application level transaction across components, it should
better be done through an orchestration process.

Kelven   


On 8/20/12 10:02 AM, "Darren Shepherd" <darren@godaddy.com> wrote:

>Murali,
>
>This is just a random implementation note.  Beware of DB transactions
>and events.  Events shouldn't be sent until DB transactions are
>committed.  If you don't give consideration to this upfront, you
>suddenly have craziness ensuing or one creates an overly complex system
>to tie DB transactions and event propagation into one framework.
>
>Darren
>
>
>
>
>> -------- Original Message --------
>> Subject: Re: [Discuss] CloudStack architecture to a loosely-coupled
>> component oriented distributed architecture
>> From: Murali Reddy <Murali.Reddy@citrix.com>
>> Date: Sun, August 19, 2012 10:29 pm
>> To: "cloudstack-dev@incubator.apache.org"
>> <cloudstack-dev@incubator.apache.org>
>> 
>> 
>> On 16/08/12 11:27 PM, "Kelven Yang" <kelven.yang@citrix.com> wrote:
>> 
>> >
>> >On 8/15/12 10:57 PM, "Alex Huang" <Alex.Huang@citrix.com> wrote:
>> >
>> >
>> >>
>> >>I don't think Murali and Kelven's proposals are related.  Murali's is
>>a
>> >>notification system so that someone who doesn't even want to know
>>about
>> >>cloudstack's code can still interface with cloudstack and react to
>>entity
>> >>changes.  Kelven's defines the ipc mechanism used by cloudstack
>> >>components.  I think it's best for these two to grow separately.  Now
>> >>it's possible that the underlying mechanism to implement these two
>>ends
>> >>up to be the same thing but at least their semantics should be
>>different.
>> >>
>> >>--Alex
>> >>
>> >
>> >
>> >As alex pointed, there are some differences between the two. I know
>>that
>> >in our current code base we use a lot of in-process listener-callback
>> >pattern to encapsulate isolated logic in different modules for better
>> >modularization. Inside the same component, this pattern applies
>>perfectly.
>> >However, when we come to a distributed world, unless the logic semantic
>> >implies that this can be decoupled in disconnected fashion, we should
>>not
>> >use the listener-callback patter across components, it assumes strong
>> >coupling, if the business logic requires such strong semantics, it
>>should
>> >go through in orchestration process so that the flow state can be
>>managed
>> >in terms of component failures.
>> >   
>> >Kelven
>> >
>> 
>> I do agree that what I am trying to do it an 'event notification system'
>> by which components external to CloudStack gets a push notification of a
>> resource state change in CloudStack, and the 'message event' proposed by
>> Kelven is more of IPC mechanism for CloudStack components to interact in
>> distributed architecture. But there are areas which seems to overlap
>> significantly which needs to be discussed.
>> 
>> Kelven, can you please elaborate a bit on 'message event' in your
>>proposal
>> which seems to main IPC between the components? I am planning to
>>introduce
>> a state machine with every virtual/physical resource that CloudStack
>> manages and generate events on the state change where subscribers are
>> external components. If your plan is to have an event-driven
>>architecture
>> where orchestration engine and network/storage/compute components
>>perform
>> a work flow with exchange of resource state change (or required state
>> change events) then same MOM (message oriented middleware) and pub-sub
>> semantics can be shared except for the topic/channel will differ. If
>>your
>> plan is IPC to be message oriented or RPC over message queue semantics
>> then at least that MOM is a common infrastructure piece on which both
>>IPC
>> and pub/sub event notifications can be built. Let me know your thoughts
>>on
>> these.
>> 
>> Redis, seems to be good messaging middleware, but is it in-memory? can
>>it
>> scale and run as cluster? How does it weigh againest AMQP as MOM
>> middleware for CloudStack?


Mime
View raw message