cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Murali Reddy <Murali.Re...@citrix.com>
Subject [MERGE] merge 'events-framework' branch to master
Date Mon, 28 Jan 2013 15:17:36 GMT
I would like to request merge of events framework [4] branch changes in to
master to branch.

I have proposed a while ago [1], about the need for a framework to
Publish-Subscribe CloudStack to events. Initially I was planning to use
simple in-memory based publish-subscribe mechanism using google event bus
library as described in the FS [3]. But considering the long term goal of
CloudStack to decouple orchestration subsystems to independently runnable
services as proposed architecture refactoring [2], and as per the
discussion followed, I have taken approach of publishing CloudStack events
to a messaging broker. Context diagram of how CloudStack and MOM (message
oriented middleware) fits in documented at [4].

Idea was to enable two use cases.

- external components like CMDB, billing systems, etc can directly get the
events from the message broker, enabling complete decoupling. Also get
instant notification of the events (like new VM create etc) to trigger
further processing.

- be able to create a notification service which can push notifications to
different end points (web call back, sms, email) line AWS SNS.

As part of this effort I am introducing a event bus abstraction which is
realized by an interface [5] providing publish, subscribe, unsubscribe
semantics. One can have a plug in that implements this interface to
provide the actually event bus implementation. CloudStack core uses the
interface to publish/subscribe to the events. CloudStack will by default
will publish all Action Events, Usage Events, Alerts, Resource State
change events. Resource state change is new type of event introduced which
gets generated on resource (Vm, volume, snapshot, network etc) state
changes.

I looked at the options to provide a implementation using the framework.
Integrating with a AMQP complaint message broker seemed better due to
availability of multiple implementations (Rabbit MQ, Active MQ, Qpid etc).
I used rabbitMQ [6] java AMQP client to interact with AMQP servers.

AMQP has a publish-subscribe paradigm. AMQP has notion of 'Exchanges' and
'Queues'. Messages are published on to the 'Exchanges' specifying a
routing key. Queue's are subscriber that bound to exchanges with a
specific binding key. AMQP implementation is responsible for delivering a
message in exchange to queue's that has the matching binding key. On
message arrival in a queue, AMQP server can send call back to consumers.

So the plug-in I have implemented using rabbit MQ, publishes events by
CloudStack on to configured exchange on AMQP server. For each of the
subscriber plug-in is responsible for creating a queue with specified
event topic.

Routing is designed to have the format.

Event-source.Event-Category.Event-Type.Resource.ResourceUUID. For e.g. A
message is published with a routing key:
management-server:ActionEvent:SNAPSHOT-CREATE:Snapshot:0a7ea29e-691b-11e2-b
afa-2c3ba27d8c47.

A subscriber interested in receiving all the events corresponding to a VM
with a UUID 9d827485-0f46-4db8-bd39-fede97cbac0c would result in a queue
with a binding key

*.*.*.VirtualMachine.9d827485-0f46-4db8-bd39-fede97cbac0c.
 
This feature is contained, does not touch business logic of the
subsystems, and hooks in to convenient classes that are used to persist
Action, Usage and Alert events in to DB. Also at run time if no plug-in is
configured that provides implementation of EventBus abstraction, there
will not have any impact of CloudStack.
 
I have added state machine to VirtualMachine, Network, Volume, Snapshot
resources, and on state transition generates a resource state change
notification. I can add port merge, if any other critical resource for
which state change event would be valuable.

'events-framework' branch is upto date with master and there are no
unit-test failures.

[1] http://markmail.org/thread/tmqagkbj7urzcouf
[2] http://markmail.org/message/dz66se5biblj5nri
[3] 
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Event+Notification+F
ramework+Proposal 
[4] 
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Event+Notification+w
ith+message+oriented+middleware+Proposal
[4] 
https://git-wip-us.apache.org/repos/asf?p=incubator-cloudstack.git;a=shortl
og;h=refs/heads/events-framework
[5] 
https://git-wip-us.apache.org/repos/asf?p=incubator-cloudstack.git;a=blob;f
=framework/events/src/org/apache/cloudstack/framework/events/EventBus.java;
h=c16ee6f96f4dfdca3a5c20e6476703c05c374df9;hb=refs/heads/events-framework

[6] http://www.rabbitmq.com/java-client.html
[


Mime
View raw message