Mailing-List: contact cloudstack-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: cloudstack-dev@incubator.apache.org
Received-SPF: pass (nike.apache.org: domain of Frank.Zhang@citrix.com
 designates 66.165.176.63 as permitted sender)
From: Frank Zhang <Frank.Zhang@citrix.com>
To: "cloudstack-dev@incubator.apache.org"
	<cloudstack-dev@incubator.apache.org>
Date: Tue, 29 Jan 2013 11:12:40 -0800
Subject: RE: [MERGE] merge 'events-framework' branch to master
Thread-Topic: [MERGE] merge 'events-framework' branch to master
Thread-Index: Ac3949boR46B9wjgSeCyJ3IVhDrdUgAcJMIg
Message-ID: 
 <93099572B72EB341B81A644E134F240B012F747FE60F@SJCPMAILBOX01.citrite.net>
References: 
 <93099572B72EB341B81A644E134F240B012F747FE607@SJCPMAILBOX01.citrite.net>
 <CD2D5E3F.2EA30%murali.reddy@citrix.com>
In-Reply-To: <CD2D5E3F.2EA30%murali.reddy@citrix.com>
Accept-Language: en-US
Content-Language: en-US
acceptlanguage: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

>=20
> >Sorry I may be late on this topic
> >
> >> Routing is designed to have the format.
> >>
> >> Event-source.Event-Category.Event-Type.Resource.ResourceUUID. For
> e.g.
> >> A message is published with a routing key:
> >> management-server:ActionEvent:SNAPSHOT-
> CREATE:Snapshot:0a7ea29e-
> >> 691b-11e2-b
> >> afa-2c3ba27d8c47.
> >>
> >> A subscriber interested in receiving all the events corresponding to
> >>a VM  with a UUID 9d827485-0f46-4db8-bd39-fede97cbac0c would result in
> >>a queue  with a binding key
> >>
> >> *.*.*.VirtualMachine.9d827485-0f46-4db8-bd39-fede97cbac0c.
> >
> >Do you mean each resource(e.g. vm, volume, snapshot ..) will have a queu=
e?
>=20
> Frank,
>=20
> Sorry, if I was not clear in my description. Queue will be per subscripti=
on and
> not for each resource. So, when a subscriber registers with event bus wit=
h
> interested topic as 'all virtual machine events corresponding to VM with
> UUID 9d827485-0f46-4db8-bd39-fede97cbac0c' then a queue is created
> which will be attached to exchange with binding key
> "*.*.*.VirtualMachine.9d827485-0f46-4db8-bd39-fede97cbac0c".

Ok. this makes more sense as I think there would not be too many subscriber=
s.
So you would notify publisher that a new queue has been created? Otherwise
how does publisher push the events out?


>=20
> -Murali
>=20
> >If so, this is not doable to Rabbitmq.  Queue is very expensive
> >resource in Rabbitmq, it's not scale to handle huge number of queues.
> >I used to do some tests on Rabbitmq to practice patterns learnt from
> >"Enterprise Integration Patterns",
> >30000 queues totally ruined Rabbitmq, it has 400M meta data on disk,
> >ate 6G memory during running.
> >And after a while Rabbitmq finally died, it hanged on starting phased
> >as it couldn't load so many queues correctly.
> >
> >Given a typical CloudStack deployment, a few thousands VM will result
> >in probably hundred thousands queues, it will finally be a big
> >performance bottleneck.
> >
> >Using queue per resource still have a disadvantage that you can't
> >subscribe resource create event, as there is no UUID used in queue
> >name.
> >
> >Instead of queue per resource, I suggest queue per resource type, and
> >encoding resource uuid in event.
> >By this way, CloudStack will only have queues less than 10.
> >
> >For example, for VM related events, there is only one queue called:
> >
> >VirutalMachine.queue
> >
> >And a few routing keys:
> >
> >VirutalMachine:ActionEvent:CREATE
> >VirutalMachine:ActionEvent:STOP
> >VirutalMachine:ActionEvent:START
> >VirutalMachine:ActionEvent:RESTART
> >VirutalMachine:ActionEvent:DESTROY
> >
> >And an event looks like:
> >
> >Class VirtualMachineEvent {
> >	private String uuid;   // encoding resource uuid in event
> >                ....
> >}
> >
> >>
> >> This feature is contained, does not touch business logic of the
> >>subsystems,  and hooks in to convenient classes that are used to
> >>persist Action, Usage and  Alert events in to DB. Also at run time if
> >>no plug-in is configured that provides  implementation of EventBus
> >>abstraction, there will not have any impact of  CloudStack.
> >>
> >> I have added state machine to VirtualMachine, Network, Volume,
> >>Snapshot  resources, and on state transition generates a resource
> >>state change  notification. I can add port merge, if any other
> >>critical resource for which  state change event would be valuable.
> >>
> >>
> >> 'events-framework' branch is upto date with master and there are no
> >>unit-
> >> test failures.
> >>
> >> [1] http://markmail.org/thread/tmqagkbj7urzcouf
> >> [2] http://markmail.org/message/dz66se5biblj5nri
> >> [3]
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Event+Notifica
> >> ti
> >> on+F
> >> ramework+Proposal
> >> [4]
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Event+Notifica
> >> ti
> >> on+w
> >> ith+message+oriented+middleware+Proposal
> >> [4]
> >>
> >>https://git-wip-us.apache.org/repos/asf?p=3Dincubator-cloudstack.git;a=
=3Ds
> >>hor
> >>tl
> >> og;h=3Drefs/heads/events-framework
> >> [5]
> >>
> >>https://git-wip-us.apache.org/repos/asf?p=3Dincubator-cloudstack.git;a=
=3Db
> >>lob
> >>;f
> >>
> =3Dframework/events/src/org/apache/cloudstack/framework/events/EventB
> >> us.java;
> >> h=3Dc16ee6f96f4dfdca3a5c20e6476703c05c374df9;hb=3Drefs/heads/events-
> >> framework
> >>
> >> [6] http://www.rabbitmq.com/java-client.html
> >> [
> >
> >
>=20