incubator-cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From George Reese <george.re...@enstratus.com>
Subject Re: Event Publish and Subscribe ( was perhaps Re: Cloudstack Questions )
Date Fri, 18 May 2012 17:14:29 GMT
We're not looking for a message queue, we're looking for a notifications system ala AWS SNS.
CloudStack needs to be able to *push* events out to endpoints; not send them to an MQ for
interested parties to pull.

-George

On May 18, 2012, at 12:09 PM, Chiradeep Vittal wrote:

> While embedding the publisher inside CloudStack could be one way to go, it is probably
more immediately useful if we push events to a popular queueing system such as AMQP? 
> Having CS have its own unique way of publishing events does make it more work for others
to utilize these events.
> Agree on having a design that allows for other systems of notifications.
> 
> From: George Reese <george.reese@enstratus.com>
> Date: Fri, 18 May 2012 05:24:42 -0700
> To: "cloudstack-dev@incubator.apache.org" <cloudstack-dev@incubator.apache.org>
> Cc: Nitin Mehta <Nitin.Mehta@citrix.com>, Kishan Kavala <Kishan.Kavala@citrix.com>,
Dean <cloud@tizatron.com>, Chiradeep Vittal <chiradeep.vittal@citrix.com>
> Subject: Re: Event Publish and Subscribe ( was perhaps Re: Cloudstack Questions )
> 
> Here's the way event pub sub should work…
> 
> People interested in events will subscribe to them via API and CloudStack will maintain
a record that includes:
> 
> * What type of event(s) the subscriber cares about
> * The URI of an endpoint to notify on events
> * An optional API key for signing event publications
> 
> If you want it to be really rich, you allow for multiple notifications endpoints, including
SMS and email.
> 
> When the existing event management system in CloudStack identifies a state change, it
published it to a message queue.
> 
> A new CloudStack component is subscribed to the message queue and pulls the state change
off. It then sends the event to all interested parties. It does not keep delivery state. It
sends and forgets. The only thing resembling state is the counting of failures, eventually
killing an endpoint that fails to often.
> 
> -George
> 
> On May 17, 2012, at 10:53 PM, Abhinandan Prateek wrote:
> 
>> Cloudstack already have a well-defined set of events, currently being used by the
Usage module. 
>> We can extend this to publish the events to registered listeners/subscribers.
>> 
>> Nitin,
>>   We should evaluate the google guava package for this purpose.
>> 
>> -abhi
>> 
>>> -----Original Message-----
>>> From: Chiradeep Vittal [mailto:Chiradeep.Vittal@citrix.com]
>>> Sent: Saturday, May 12, 2012 3:48 AM
>>> To: cloudstack-dev@incubator.apache.org
>>> Cc: Dean
>>> Subject: Re: Event Publish and Subscribe ( was perhaps Re: Cloudstack
>>> Questions )
>>> 
>>> +1.
>>> This could also be used for example to update DNS records on a DNS server
>>> whenever a VM is started / stopped in a network.
>>> A subscriber to the event bus can do this, and potentially things like updating
a
>>> CMDB or an LDAP server.
>>> 
>>> The important caveat is that this will be asynchronous to the stuff happening
>>> inside the management server: it has limited utility for doing stuff that  is
>>> required to succeed before an API operation can succeed.
>>> For example, if a VM start requires a NAT rule to be programmed on the
>>> firewall before the start can be considered successful. You could write a event
>>> bus subscriber that does this, but the management server would not know if
>>> it was successful or not.
>>> 
>>> There's at least a couple of entry points:
>>> 1. com.cloud.utils.fsm.StateListener. Classes that implement this listener get
>>> notified on finite state event transitions. Examples include VM start / stop
2.
>>> com.cloud.network.element.NetworkElement. Classes that implement this
>>> get notified of network state transitions. Examples include adding nics and
>>> removing nics to/from a network.
>>> 
>>> I think the first question regarding monitoring VMs on the isolated VLAN had
a
>>> different origin though.
>>> The intention there was to have a monitoring service (e.g., Nagios) reach into
>>> the VM and monitor stuff like CPU, IO, or even applications.
>>> The issue there was around network reachability.
>>> 
>>> --
>>> Chiradeep
>>> 
>>> 
>>> 
>>> On 5/11/12 2:09 PM, "Adrian Cole" <adrian@jclouds.org> wrote:
>>> 
>>>> +1
>>>> 
>>>> Makes sense to have pubsub.  Inside the java codebase, we could
>>>> consider a clean and idiomatic lib like guava which is easy to unit test.
>>>> 
>>>> http://codingjunkie.net/guava-eventbus/
>>>> 
>>>> Then, expose out-of-JVM hooks for any of the popular services people use.
>>>> 
>>>> -A
>>>> On May 11, 2012 1:58 PM, "Dean" <cloud@tizatron.com> wrote:
>>>> 
>>>>> Cross reference to:
>>>>> 
>>>>> 
>>>>> 
>>>>> http://mail-archives.apache.org/mod_mbox/incubator-cloudstack-
>>> dev/201204.
>>>>> mbox/browser
>>>>> 
>>>>> [ from: Marlon Davids ]
>>>>> < munch >
>>>>>> 2) How do we monitor VM's that are in Cloudstack when they are in
>>>>>> an
>>>>> isolated VLAN does
>>>>>> anyone have a clever workaround?
>>>>>> 3) Has anyone developed a script for parsing and alerting on
>>>>>> warning
>>>>> events in the
>>>>>> management Log yet?
>>>>> 
>>>>> I would like to propose cloudstack consider a pub/sub model for event
>>>>> handling to complement API calls like listEvents.
>>>>> 
>>>>> Polling can be problematic and sensitive to scaling.
>>>>> 
>>>>> A simple example would be state change on a physical device.  The
>>>>> admin  server can simply publish a message on a network socket
>>>>> indicating that the  device has changed it's state.
>>>>> 
>>>>> If a subscriber was interested in that device, it could make an api
>>>>> call  to the admin server for state change information for that device
only.
>>>>> The
>>>>> admin server may choose to validate that physical device against the
>>>>> current state table in the database.
>>>>> 
>>>>> The API would reply that this node changed it's state from
>>>>> Operational to  Prep For Maintenance.  (or whatever the transition
>>>>> state would be)
>>>>> 
>>>>> The message exchange could be wrapped around vm states, resource
>>>>> additions/removals etc.
>>>>> 
>>>>> Using a library like zeromq, a developer can write any number of
>>>>> consumers  in any language they wanted to subscribe to the Event Bus.
>>>>> 
>>>>> Comments?
>>>>> 
>>>>> --
>>>>> -Dean
>> 
> 
> --
> George Reese - Chief Technology Officer, enStratus
> e: george.reese@enstratus.com    Skype: nspollution    t: @GeorgeReese    p: +1.207.956.0217
> enStratus: Enterprise Cloud Management - @enStratus - http://www.enstratus.com
> To schedule a meeting with me: http://tungle.me/GeorgeReese
> 

--
George Reese - Chief Technology Officer, enStratus
e: george.reese@enstratus.com    Skype: nspollution    t: @GeorgeReese    p: +1.207.956.0217
enStratus: Enterprise Cloud Management - @enStratus - http://www.enstratus.com
To schedule a meeting with me: http://tungle.me/GeorgeReese


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message