Mailing-List: contact cloudstack-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: cloudstack-dev@incubator.apache.org
Received-SPF: pass (nike.apache.org: domain of Murali.Reddy@citrix.com
 designates 203.166.19.134 as permitted sender)
From: Murali Reddy <Murali.Reddy@citrix.com>
To: "cloudstack-dev@incubator.apache.org"
	<cloudstack-dev@incubator.apache.org>
Date: Tue, 29 Jan 2013 11:15:25 +0530
Subject: Re: [MERGE] merge 'events-framework' branch to master
Thread-Topic: [MERGE] merge 'events-framework' branch to master
Thread-Index: Ac3949boR46B9wjgSeCyJ3IVhDrdUg==
Message-ID: <CD2D5E3F.2EA30%murali.reddy@citrix.com>
In-Reply-To: 
 <93099572B72EB341B81A644E134F240B012F747FE607@SJCPMAILBOX01.citrite.net>
Accept-Language: en-US
Content-Language: en-US
user-agent: Microsoft-MacOutlook/14.2.5.121010
acceptlanguage: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

On 29/01/13 12:37 AM, "Frank Zhang" <Frank.Zhang@citrix.com> wrote:

>Sorry I may be late on this topic
>
>> Routing is designed to have the format.
>>=20
>> Event-source.Event-Category.Event-Type.Resource.ResourceUUID. For e.g.
>> A message is published with a routing key:
>> management-server:ActionEvent:SNAPSHOT-CREATE:Snapshot:0a7ea29e-
>> 691b-11e2-b
>> afa-2c3ba27d8c47.
>>=20
>> A subscriber interested in receiving all the events corresponding to a
>>VM
>> with a UUID 9d827485-0f46-4db8-bd39-fede97cbac0c would result in a queue
>> with a binding key
>>=20
>> *.*.*.VirtualMachine.9d827485-0f46-4db8-bd39-fede97cbac0c.
>
>Do you mean each resource(e.g. vm, volume, snapshot ..) will have a queue?

Frank,

Sorry, if I was not clear in my description. Queue will be per
subscription and not for each resource. So, when a subscriber registers
with event bus with interested topic as 'all virtual machine events
corresponding to VM with UUID 9d827485-0f46-4db8-bd39-fede97cbac0c' then a
queue is created which will be attached to exchange with binding key
"*.*.*.VirtualMachine.9d827485-0f46-4db8-bd39-fede97cbac0c".

-Murali

>If so, this is not doable to Rabbitmq.  Queue is very expensive resource
>in Rabbitmq,
>it's not scale to handle huge number of queues.
>I used to do some tests on Rabbitmq to practice patterns learnt from
>"Enterprise Integration Patterns",
>30000 queues totally ruined Rabbitmq, it has 400M meta data on disk, ate
>6G memory during running.
>And after a while Rabbitmq finally died, it hanged on starting phased as
>it couldn't load so many queues
>correctly.
>
>Given a typical CloudStack deployment, a few thousands VM will result in
>probably hundred thousands
>queues, it will finally be a big performance bottleneck.
>
>Using queue per resource still have a disadvantage that you can't
>subscribe resource create event, as there
>is no UUID used in queue name.
>
>Instead of queue per resource, I suggest queue per resource type, and
>encoding resource uuid in event.
>By this way, CloudStack will only have queues less than 10.
>
>For example, for VM related events, there is only one queue called:
>
>VirutalMachine.queue
>
>And a few routing keys:
>
>VirutalMachine:ActionEvent:CREATE
>VirutalMachine:ActionEvent:STOP
>VirutalMachine:ActionEvent:START
>VirutalMachine:ActionEvent:RESTART
>VirutalMachine:ActionEvent:DESTROY
>
>And an event looks like:
>
>Class VirtualMachineEvent {
>	private String uuid;   // encoding resource uuid in event
>                ....
>}
>
>>=20
>> This feature is contained, does not touch business logic of the
>>subsystems,
>> and hooks in to convenient classes that are used to persist Action,
>>Usage and
>> Alert events in to DB. Also at run time if no plug-in is configured
>>that provides
>> implementation of EventBus abstraction, there will not have any impact
>>of
>> CloudStack.
>>=20
>> I have added state machine to VirtualMachine, Network, Volume, Snapshot
>> resources, and on state transition generates a resource state change
>> notification. I can add port merge, if any other critical resource for
>>which
>> state change event would be valuable.
>>=20
>>=20
>> 'events-framework' branch is upto date with master and there are no
>>unit-
>> test failures.
>>=20
>> [1] http://markmail.org/thread/tmqagkbj7urzcouf
>> [2] http://markmail.org/message/dz66se5biblj5nri
>> [3]
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Event+Notificati
>> on+F
>> ramework+Proposal
>> [4]
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Event+Notificati
>> on+w
>> ith+message+oriented+middleware+Proposal
>> [4]
>>=20
>>https://git-wip-us.apache.org/repos/asf?p=3Dincubator-cloudstack.git;a=3D=
shor
>>tl
>> og;h=3Drefs/heads/events-framework
>> [5]
>>=20
>>https://git-wip-us.apache.org/repos/asf?p=3Dincubator-cloudstack.git;a=3D=
blob
>>;f
>> =3Dframework/events/src/org/apache/cloudstack/framework/events/EventB
>> us.java;
>> h=3Dc16ee6f96f4dfdca3a5c20e6476703c05c374df9;hb=3Drefs/heads/events-
>> framework
>>=20
>> [6] http://www.rabbitmq.com/java-client.html
>> [
>
>