Mailing-List: contact cloudstack-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: cloudstack-dev@incubator.apache.org
Received-SPF: pass (nike.apache.org: domain of darren@godaddy.com designates
 208.109.78.113 as permitted sender)
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset="utf-8"
User-Agent: Workspace Webmail 5.6.24
Message-Id: 
 <20120820100235.d3d18d9a633cb81ed61112bf108fc615.3167cc9c8d.wbe@email00.secureserver.net>
From: "Darren Shepherd" <darren@godaddy.com>
To: cloudstack-dev@incubator.apache.org
Cc: Murali.Reddy@citrix.com
Subject: RE: [Discuss] CloudStack architecture to a loosely-coupled component
 oriented distributed architecture
Date: Mon, 20 Aug 2012 10:02:35 -0700
Mime-Version: 1.0

Murali,=0A=0AThis is just a random implementation note.  Beware of DB trans=
actions=0Aand events.  Events shouldn't be sent until DB transactions are=
=0Acommitted.  If you don't give consideration to this upfront, you=0Asudde=
nly have craziness ensuing or one creates an overly complex system=0Ato tie=
 DB transactions and event propagation into one framework.=0A=0ADarren=0A=
=0A=0A=0A=0A> -------- Original Message --------=0A> Subject: Re: [Discuss]=
 CloudStack architecture to a loosely-coupled=0A> component oriented distri=
buted architecture=0A> From: Murali Reddy <Murali.Reddy@citrix.com>=0A> Dat=
e: Sun, August 19, 2012 10:29 pm=0A> To: "cloudstack-dev@incubator.apache.o=
rg"=0A> <cloudstack-dev@incubator.apache.org>=0A> =0A> =0A> On 16/08/12 11:=
27 PM, "Kelven Yang" <kelven.yang@citrix.com> wrote:=0A> =0A> >=0A> >On 8/1=
5/12 10:57 PM, "Alex Huang" <Alex.Huang@citrix.com> wrote:=0A> >=0A> >=0A> =
>>=0A> >>I don't think Murali and Kelven's proposals are related.  Murali's=
 is a=0A> >>notification system so that someone who doesn't even want to kn=
ow about=0A> >>cloudstack's code can still interface with cloudstack and re=
act to entity=0A> >>changes.  Kelven's defines the ipc mechanism used by cl=
oudstack=0A> >>components.  I think it's best for these two to grow separat=
ely.  Now=0A> >>it's possible that the underlying mechanism to implement th=
ese two ends=0A> >>up to be the same thing but at least their semantics sho=
uld be different.=0A> >>=0A> >>--Alex=0A> >>=0A> >=0A> >=0A> >As alex point=
ed, there are some differences between the two. I know that=0A> >in our cur=
rent code base we use a lot of in-process listener-callback=0A> >pattern to=
 encapsulate isolated logic in different modules for better=0A> >modulariza=
tion. Inside the same component, this pattern applies perfectly.=0A> >Howev=
er, when we come to a distributed world, unless the logic semantic=0A> >imp=
lies that this can be decoupled in disconnected fashion, we should not=0A> =
>use the listener-callback patter across components, it assumes strong=0A> =
>coupling, if the business logic requires such strong semantics, it should=
=0A> >go through in orchestration process so that the flow state can be man=
aged=0A> >in terms of component failures.=0A> >   =0A> >Kelven=0A> >=0A> =
=0A> I do agree that what I am trying to do it an 'event notification syste=
m'=0A> by which components external to CloudStack gets a push notification =
of a=0A> resource state change in CloudStack, and the 'message event' propo=
sed by=0A> Kelven is more of IPC mechanism for CloudStack components to int=
eract in=0A> distributed architecture. But there are areas which seems to o=
verlap=0A> significantly which needs to be discussed.=0A> =0A> Kelven, can =
you please elaborate a bit on 'message event' in your proposal=0A> which se=
ems to main IPC between the components? I am planning to introduce=0A> a st=
ate machine with every virtual/physical resource that CloudStack=0A> manage=
s and generate events on the state change where subscribers are=0A> externa=
l components. If your plan is to have an event-driven architecture=0A> wher=
e orchestration engine and network/storage/compute components perform=0A> a=
 work flow with exchange of resource state change (or required state=0A> ch=
ange events) then same MOM (message oriented middleware) and pub-sub=0A> se=
mantics can be shared except for the topic/channel will differ. If your=0A>=
 plan is IPC to be message oriented or RPC over message queue semantics=0A>=
 then at least that MOM is a common infrastructure piece on which both IPC=
=0A> and pub/sub event notifications can be built. Let me know your thought=
s on=0A> these.=0A> =0A> Redis, seems to be good messaging middleware, but =
is it in-memory? can it=0A> scale and run as cluster? How does it weigh aga=
inest AMQP as MOM=0A> middleware for CloudStack?