incubator-cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kelven Yang <>
Subject Re: [Discuss] CloudStack architecture to a loosely-coupled component oriented distributed architecture
Date Mon, 20 Aug 2012 17:53:30 GMT

The key point is the shifting to event-driven programming model itself at
orchestration layer, the underlying facility is currently not at critical
path, we will need to do a messaging arbitration here so that we can
plugin a suitable MQ selection in a fairly easy step. One thing I'm sure
is that we won't use two separated MQ solutions in one system.

We also need to think twice of tight-coupled listener pattern across
components, if listeners from different components are implicitly required
to participate application level transaction across components, it should
better be done through an orchestration process.


On 8/20/12 10:02 AM, "Darren Shepherd" <> wrote:

>This is just a random implementation note.  Beware of DB transactions
>and events.  Events shouldn't be sent until DB transactions are
>committed.  If you don't give consideration to this upfront, you
>suddenly have craziness ensuing or one creates an overly complex system
>to tie DB transactions and event propagation into one framework.
>> -------- Original Message --------
>> Subject: Re: [Discuss] CloudStack architecture to a loosely-coupled
>> component oriented distributed architecture
>> From: Murali Reddy <>
>> Date: Sun, August 19, 2012 10:29 pm
>> To: ""
>> <>
>> On 16/08/12 11:27 PM, "Kelven Yang" <> wrote:
>> >
>> >On 8/15/12 10:57 PM, "Alex Huang" <> wrote:
>> >
>> >
>> >>
>> >>I don't think Murali and Kelven's proposals are related.  Murali's is
>> >>notification system so that someone who doesn't even want to know
>> >>cloudstack's code can still interface with cloudstack and react to
>> >>changes.  Kelven's defines the ipc mechanism used by cloudstack
>> >>components.  I think it's best for these two to grow separately.  Now
>> >>it's possible that the underlying mechanism to implement these two
>> >>up to be the same thing but at least their semantics should be
>> >>
>> >>--Alex
>> >>
>> >
>> >
>> >As alex pointed, there are some differences between the two. I know
>> >in our current code base we use a lot of in-process listener-callback
>> >pattern to encapsulate isolated logic in different modules for better
>> >modularization. Inside the same component, this pattern applies
>> >However, when we come to a distributed world, unless the logic semantic
>> >implies that this can be decoupled in disconnected fashion, we should
>> >use the listener-callback patter across components, it assumes strong
>> >coupling, if the business logic requires such strong semantics, it
>> >go through in orchestration process so that the flow state can be
>> >in terms of component failures.
>> >   
>> >Kelven
>> >
>> I do agree that what I am trying to do it an 'event notification system'
>> by which components external to CloudStack gets a push notification of a
>> resource state change in CloudStack, and the 'message event' proposed by
>> Kelven is more of IPC mechanism for CloudStack components to interact in
>> distributed architecture. But there are areas which seems to overlap
>> significantly which needs to be discussed.
>> Kelven, can you please elaborate a bit on 'message event' in your
>> which seems to main IPC between the components? I am planning to
>> a state machine with every virtual/physical resource that CloudStack
>> manages and generate events on the state change where subscribers are
>> external components. If your plan is to have an event-driven
>> where orchestration engine and network/storage/compute components
>> a work flow with exchange of resource state change (or required state
>> change events) then same MOM (message oriented middleware) and pub-sub
>> semantics can be shared except for the topic/channel will differ. If
>> plan is IPC to be message oriented or RPC over message queue semantics
>> then at least that MOM is a common infrastructure piece on which both
>> and pub/sub event notifications can be built. Let me know your thoughts
>> these.
>> Redis, seems to be good messaging middleware, but is it in-memory? can
>> scale and run as cluster? How does it weigh againest AMQP as MOM
>> middleware for CloudStack?

View raw message