incubator-deltacloud-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Carl Trieloff <>
Subject Initial boot / post boot Matahari cloud agent and Deltacloud interactions
Date Fri, 17 Dec 2010 14:22:52 GMT

Based on some discussion I have worked up an additional agent with ted (some
input on qmfv2) for Matahari to be able to generically deal with 
injecting configuration
into an instance in a public or private cloud.

I'm interested in this so that we use a generic boot image and then 
provide a
piece of CDL 'Content discription Language' and specialize this generic 
to represent my application deployment.

The generic mechanism does not type the configuration for CDL, anything from
foobar to CDL to Puppet Script could be passed. First I'll discribe the 
and then I'll discussed the requirement from this which I have on 
DeltaCloud API.

I've implemented a mechanism that can be used/ deployed in one of the 
following ways.

a.) Federated, using an instance in the cloud as a rendezvous point, 
With explicit instance
selection. Does not require inbound connections to source
b.) Federated or non Federated without an instance in the cloud, With 
explicit instance
selection.. Requires in-bound connections to source
c.) I have also created non explicit instance selection / for the above 
two deployment

Let me explain the details of each:

The flow of events in (a & b) are as follows.

- Instantiate a cache on-premise, or in my service.
- Start one or more instances in the cloud (EC2) for example as a local 
cache proxy.
- Federate the two caches
(b) skips the last two steps...

- On Instance start request provide
         - a config uuid
         - the CDL / configuration for the node for post boot.
         - a secrete & the IP of cache is auto prefilled
         - This function does two things
             1.) Start the  instance passing uuid:IP:secrete as a boot 
param to the image for the post boot daemon.
             2.) Calls pre_register() on the cache with
                 - uuid
                 - CDL
                 - secrete.

- The instance now starts, and the Manatahi config daemon takes the IP, 
connects to
the rendezvous point looks up the pre_registered object from the cache 
and downloads
the CDL/ conf.

- After configuring itself and completing the boot, it then calls 
update_cache() passing the
uuid, secrete, para_map, ip and status.

- The origin service can now retrieve the IP of the booted node, any 
param it set from the map
and the instance status.

- It is also possible to provide updates of config/ CDL to the running 
instance if required.

(These two deployment options require that a boot string be passed into 
the cloud with the
following information uuid:IP:secrete ) via the Delta cloud API. The 
DeltaCloud API does
not need to care about the format, rather just pass a small string onto 
the instance.


For flow (c). interface is build however the implementation is not fully 
coded as I don't think
the use case is so strong.

In this case, we don't want to pass anything to the cloud via the API, 
or the cloud does not
support that and we need another option (eg GoGrid)

In this case, the booting instance has no uuid. The IP's of the 
rendezvous points need to be
included in the starter image.

The flow is as follows.
-  call pre_register on the cache, but mark the cache element as dynamic
- start instance in cloud
- instance in cloud contacts rendezvous point
- The cache will now hand out the uuid and config to the booting 
instances on a first come
first serve basis.
- The instance then updates the cache info as before.

- It is possible to have dynamic and non dynamic elements in the cache 
at the same time

This flow requires nothing to be passed via the deltacloudAPI (or the 
cloud does not support it),
it also means however that it is not really possible to map differing 
machine profiles to specific
configurations unless artificial constraints are used like building a 
different starter image for
each profile of machine and tagging the starter image.

Personally I prefer the flow of (a & b) but thought I would provide both.

It is implemented with QMF from the Qpid project. I will post the patch 
of the initial implementation
to the Matahari list.

Here is the schema for the service:

<!-- boot config /config & param registory-->
<class name="Config">
<property name="uuid"          type="sstr" access="RO" desc="Host UUID" 
index="y" />
<property name="config_xml"    type="lstr" access="RO" desc="Host 
configuration XML" />
<property name="my_conf_addr"  type="sstr" access="RW" desc="Address for 
this machine config" />
<property name="status"        type="sstr" access="RW" desc="Status of 
Instance" />
<property name="parameter_map" type="map"  access="RW" desc="Host 
configuration metadata" />
<property name="config_hosts"  type="list" access="RW" desc="List of my 
Configuration hosts" />
<property name="dymanic"       type="bool" access="RO" desc="Dynamically 
assign this instance" />

<class name="ConfigHelper">
<method name="pre_register"      desc="Associate a configuration with a 
<arg name="uuid"               dir="I"  type="sstr" />
<arg name="config_xml"         dir="I"  type="lstr" />
<arg name="domain"             dir="I"  type="sstr" />

<method name="update_cache"  desc="Update the instance fields in the cache">
<arg name="uuid"               dir="I"  type="sstr" />
<arg name="my_conf_addr"       dir="I"  type="sstr" />
<arg name="parameter_map"      dir="I"  type="map" />
<arg name="status"             dir="I"  type="sstr" />
<arg name="trust_key"          dir="I"  type="sstr" />

<method name="remove"         desc="Delate configuartion for UUID">
<arg name="uuid"           dir="I"  type="sstr" />

<method name="remove_stale"           desc="Delate stale configuartions">
<arg name="timeout"               dir="I"  type="uint32" />
<arg name="removed"               dir="O"  type="uint32" />

<method name="assign_dynamic"      desc="Assign dynamic instance">
<arg name="uuid"                    dir="O"  type="sstr" />


View raw message