incubator-cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Frank Zhang <Frank.Zh...@citrix.com>
Subject RE: [MERGE] merge 'events-framework' branch to master
Date Tue, 29 Jan 2013 19:12:40 GMT
> 
> >Sorry I may be late on this topic
> >
> >> Routing is designed to have the format.
> >>
> >> Event-source.Event-Category.Event-Type.Resource.ResourceUUID. For
> e.g.
> >> A message is published with a routing key:
> >> management-server:ActionEvent:SNAPSHOT-
> CREATE:Snapshot:0a7ea29e-
> >> 691b-11e2-b
> >> afa-2c3ba27d8c47.
> >>
> >> A subscriber interested in receiving all the events corresponding to
> >>a VM  with a UUID 9d827485-0f46-4db8-bd39-fede97cbac0c would result in
> >>a queue  with a binding key
> >>
> >> *.*.*.VirtualMachine.9d827485-0f46-4db8-bd39-fede97cbac0c.
> >
> >Do you mean each resource(e.g. vm, volume, snapshot ..) will have a queue?
> 
> Frank,
> 
> Sorry, if I was not clear in my description. Queue will be per subscription and
> not for each resource. So, when a subscriber registers with event bus with
> interested topic as 'all virtual machine events corresponding to VM with
> UUID 9d827485-0f46-4db8-bd39-fede97cbac0c' then a queue is created
> which will be attached to exchange with binding key
> "*.*.*.VirtualMachine.9d827485-0f46-4db8-bd39-fede97cbac0c".

Ok. this makes more sense as I think there would not be too many subscribers.
So you would notify publisher that a new queue has been created? Otherwise
how does publisher push the events out?


> 
> -Murali
> 
> >If so, this is not doable to Rabbitmq.  Queue is very expensive
> >resource in Rabbitmq, it's not scale to handle huge number of queues.
> >I used to do some tests on Rabbitmq to practice patterns learnt from
> >"Enterprise Integration Patterns",
> >30000 queues totally ruined Rabbitmq, it has 400M meta data on disk,
> >ate 6G memory during running.
> >And after a while Rabbitmq finally died, it hanged on starting phased
> >as it couldn't load so many queues correctly.
> >
> >Given a typical CloudStack deployment, a few thousands VM will result
> >in probably hundred thousands queues, it will finally be a big
> >performance bottleneck.
> >
> >Using queue per resource still have a disadvantage that you can't
> >subscribe resource create event, as there is no UUID used in queue
> >name.
> >
> >Instead of queue per resource, I suggest queue per resource type, and
> >encoding resource uuid in event.
> >By this way, CloudStack will only have queues less than 10.
> >
> >For example, for VM related events, there is only one queue called:
> >
> >VirutalMachine.queue
> >
> >And a few routing keys:
> >
> >VirutalMachine:ActionEvent:CREATE
> >VirutalMachine:ActionEvent:STOP
> >VirutalMachine:ActionEvent:START
> >VirutalMachine:ActionEvent:RESTART
> >VirutalMachine:ActionEvent:DESTROY
> >
> >And an event looks like:
> >
> >Class VirtualMachineEvent {
> >	private String uuid;   // encoding resource uuid in event
> >                ....
> >}
> >
> >>
> >> This feature is contained, does not touch business logic of the
> >>subsystems,  and hooks in to convenient classes that are used to
> >>persist Action, Usage and  Alert events in to DB. Also at run time if
> >>no plug-in is configured that provides  implementation of EventBus
> >>abstraction, there will not have any impact of  CloudStack.
> >>
> >> I have added state machine to VirtualMachine, Network, Volume,
> >>Snapshot  resources, and on state transition generates a resource
> >>state change  notification. I can add port merge, if any other
> >>critical resource for which  state change event would be valuable.
> >>
> >>
> >> 'events-framework' branch is upto date with master and there are no
> >>unit-
> >> test failures.
> >>
> >> [1] http://markmail.org/thread/tmqagkbj7urzcouf
> >> [2] http://markmail.org/message/dz66se5biblj5nri
> >> [3]
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Event+Notifica
> >> ti
> >> on+F
> >> ramework+Proposal
> >> [4]
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Event+Notifica
> >> ti
> >> on+w
> >> ith+message+oriented+middleware+Proposal
> >> [4]
> >>
> >>https://git-wip-us.apache.org/repos/asf?p=incubator-cloudstack.git;a=s
> >>hor
> >>tl
> >> og;h=refs/heads/events-framework
> >> [5]
> >>
> >>https://git-wip-us.apache.org/repos/asf?p=incubator-cloudstack.git;a=b
> >>lob
> >>;f
> >>
> =framework/events/src/org/apache/cloudstack/framework/events/EventB
> >> us.java;
> >> h=c16ee6f96f4dfdca3a5c20e6476703c05c374df9;hb=refs/heads/events-
> >> framework
> >>
> >> [6] http://www.rabbitmq.com/java-client.html
> >> [
> >
> >
> 


Mime
View raw message