incubator-deltacloud-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Carl Trieloff <>
Subject Re: Initial boot / post boot Matahari cloud agent and Deltacloud interactions
Date Fri, 17 Dec 2010 17:13:26 GMT

Copied both lists, as FYI for deltacloud to serve as use case for 
passing small
opaque data to an instance being requested to be started.


On 12/17/2010 12:11 PM, Benjamin Browning wrote:
> This is the first time I've seen anything about Matahari come up on the Deltacloud list.
Was this sent to the wrong list or is there some prior conversation about Matahari and Deltacloud
that I'm missing?
> Ben
> On Dec 17, 2010, at 9:22 AM, Carl Trieloff wrote:
>> Based on some discussion I have worked up an additional agent with ted (some
>> input on qmfv2) for Matahari to be able to generically deal with injecting configuration
>> into an instance in a public or private cloud.
>> I'm interested in this so that we use a generic boot image and then provide a
>> piece of CDL 'Content discription Language' and specialize this generic image
>> to represent my application deployment.
>> The generic mechanism does not type the configuration for CDL, anything from
>> foobar to CDL to Puppet Script could be passed. First I'll discribe the mechanics
>> and then I'll discussed the requirement from this which I have on DeltaCloud API.
>> I've implemented a mechanism that can be used/ deployed in one of the following ways.
>> a.) Federated, using an instance in the cloud as a rendezvous point, With explicit
>> selection. Does not require inbound connections to source
>> b.) Federated or non Federated without an instance in the cloud, With explicit instance
>> selection.. Requires in-bound connections to source
>> c.) I have also created non explicit instance selection / for the above two deployment
>> patterns.
>> Let me explain the details of each:
>> The flow of events in (a&  b) are as follows.
>> Setup:
>> - Instantiate a cache on-premise, or in my service.
>> - Start one or more instances in the cloud (EC2) for example as a local cache proxy.
>> - Federate the two caches
>> (b) skips the last two steps...
>> Operation
>> - On Instance start request provide
>>         - a config uuid
>>         - the CDL / configuration for the node for post boot.
>>         - a secrete&  the IP of cache is auto prefilled
>>         - This function does two things
>>             1.) Start the  instance passing uuid:IP:secrete as a boot param to the
image for the post boot daemon.
>>             2.) Calls pre_register() on the cache with
>>                 - uuid
>>                 - CDL
>>                 - secrete.
>> - The instance now starts, and the Manatahi config daemon takes the IP, connects
>> the rendezvous point looks up the pre_registered object from the cache and downloads
>> the CDL/ conf.
>> - After configuring itself and completing the boot, it then calls update_cache()
passing the
>> uuid, secrete, para_map, ip and status.
>> - The origin service can now retrieve the IP of the booted node, any param it set
from the map
>> and the instance status.
>> - It is also possible to provide updates of config/ CDL to the running instance if
>> (These two deployment options require that a boot string be passed into the cloud
with the
>> following information uuid:IP:secrete ) via the Delta cloud API. The DeltaCloud API
>> not need to care about the format, rather just pass a small string onto the instance.
>> .....................
>> For flow (c). interface is build however the implementation is not fully coded as
I don't think
>> the use case is so strong.
>> In this case, we don't want to pass anything to the cloud via the API, or the cloud
does not
>> support that and we need another option (eg GoGrid)
>> In this case, the booting instance has no uuid. The IP's of the rendezvous points
need to be
>> included in the starter image.
>> The flow is as follows.
>> -  call pre_register on the cache, but mark the cache element as dynamic
>> - start instance in cloud
>> - instance in cloud contacts rendezvous point
>> - The cache will now hand out the uuid and config to the booting instances on a first
>> first serve basis.
>> - The instance then updates the cache info as before.
>> - It is possible to have dynamic and non dynamic elements in the cache at the same
>> This flow requires nothing to be passed via the deltacloudAPI (or the cloud does
not support it),
>> it also means however that it is not really possible to map differing machine profiles
to specific
>> configurations unless artificial constraints are used like building a different starter
image for
>> each profile of machine and tagging the starter image.
>> Personally I prefer the flow of (a&  b) but thought I would provide both.
>> It is implemented with QMF from the Qpid project. I will post the patch of the initial
>> to the Matahari list.
>> Here is the schema for the service:
>> <!-- boot config /config&  param registory-->
>> <class name="Config">
>> <property name="uuid"          type="sstr" access="RO" desc="Host UUID" index="y"
>> <property name="config_xml"    type="lstr" access="RO" desc="Host configuration
XML" />
>> <property name="my_conf_addr"  type="sstr" access="RW" desc="Address for this
machine config" />
>> <property name="status"        type="sstr" access="RW" desc="Status of Instance"
>> <property name="parameter_map" type="map"  access="RW" desc="Host configuration
metadata" />
>> <property name="config_hosts"  type="list" access="RW" desc="List of my Configuration
hosts" />
>> <property name="dymanic"       type="bool" access="RO" desc="Dynamically assign
this instance" />
>> </class>
>> <class name="ConfigHelper">
>> <method name="pre_register"      desc="Associate a configuration with a UUID">
>> <arg name="uuid"               dir="I"  type="sstr" />
>> <arg name="config_xml"         dir="I"  type="lstr" />
>> <arg name="domain"             dir="I"  type="sstr" />
>> </method>
>> <method name="update_cache"  desc="Update the instance fields in the cache">
>> <arg name="uuid"               dir="I"  type="sstr" />
>> <arg name="my_conf_addr"       dir="I"  type="sstr" />
>> <arg name="parameter_map"      dir="I"  type="map" />
>> <arg name="status"             dir="I"  type="sstr" />
>> <arg name="trust_key"          dir="I"  type="sstr" />
>> </method>
>> <method name="remove"         desc="Delate configuartion for UUID">
>> <arg name="uuid"           dir="I"  type="sstr" />
>> </method>
>> <method name="remove_stale"           desc="Delate stale configuartions">
>> <arg name="timeout"               dir="I"  type="uint32" />
>> <arg name="removed"               dir="O"  type="uint32" />
>> </method>
>> <method name="assign_dynamic"      desc="Assign dynamic instance">
>> <arg name="uuid"                    dir="O"  type="sstr" />
>> </method>
>> </class>

View raw message