cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Murali Reddy <>
Subject Re: [MERGE] merge 'events-framework' branch to master
Date Tue, 29 Jan 2013 05:45:25 GMT
On 29/01/13 12:37 AM, "Frank Zhang" <> wrote:

>Sorry I may be late on this topic
>> Routing is designed to have the format.
>> Event-source.Event-Category.Event-Type.Resource.ResourceUUID. For e.g.
>> A message is published with a routing key:
>> management-server:ActionEvent:SNAPSHOT-CREATE:Snapshot:0a7ea29e-
>> 691b-11e2-b
>> afa-2c3ba27d8c47.
>> A subscriber interested in receiving all the events corresponding to a
>> with a UUID 9d827485-0f46-4db8-bd39-fede97cbac0c would result in a queue
>> with a binding key
>> *.*.*.VirtualMachine.9d827485-0f46-4db8-bd39-fede97cbac0c.
>Do you mean each resource(e.g. vm, volume, snapshot ..) will have a queue?


Sorry, if I was not clear in my description. Queue will be per
subscription and not for each resource. So, when a subscriber registers
with event bus with interested topic as 'all virtual machine events
corresponding to VM with UUID 9d827485-0f46-4db8-bd39-fede97cbac0c' then a
queue is created which will be attached to exchange with binding key


>If so, this is not doable to Rabbitmq.  Queue is very expensive resource
>in Rabbitmq,
>it's not scale to handle huge number of queues.
>I used to do some tests on Rabbitmq to practice patterns learnt from
>"Enterprise Integration Patterns",
>30000 queues totally ruined Rabbitmq, it has 400M meta data on disk, ate
>6G memory during running.
>And after a while Rabbitmq finally died, it hanged on starting phased as
>it couldn't load so many queues
>Given a typical CloudStack deployment, a few thousands VM will result in
>probably hundred thousands
>queues, it will finally be a big performance bottleneck.
>Using queue per resource still have a disadvantage that you can't
>subscribe resource create event, as there
>is no UUID used in queue name.
>Instead of queue per resource, I suggest queue per resource type, and
>encoding resource uuid in event.
>By this way, CloudStack will only have queues less than 10.
>For example, for VM related events, there is only one queue called:
>And a few routing keys:
>And an event looks like:
>Class VirtualMachineEvent {
>	private String uuid;   // encoding resource uuid in event
>                ....
>> This feature is contained, does not touch business logic of the
>> and hooks in to convenient classes that are used to persist Action,
>>Usage and
>> Alert events in to DB. Also at run time if no plug-in is configured
>>that provides
>> implementation of EventBus abstraction, there will not have any impact
>> CloudStack.
>> I have added state machine to VirtualMachine, Network, Volume, Snapshot
>> resources, and on state transition generates a resource state change
>> notification. I can add port merge, if any other critical resource for
>> state change event would be valuable.
>> 'events-framework' branch is upto date with master and there are no
>> test failures.
>> [1]
>> [2]
>> [3]
>> on+F
>> ramework+Proposal
>> [4]
>> on+w
>> ith+message+oriented+middleware+Proposal
>> [4]
>> og;h=refs/heads/events-framework
>> [5]
>> =framework/events/src/org/apache/cloudstack/framework/events/EventB
>> h=c16ee6f96f4dfdca3a5c20e6476703c05c374df9;hb=refs/heads/events-
>> framework
>> [6]
>> [

View raw message