incubator-deltacloud-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "marios@redhat.com" <mandr...@redhat.com>
Subject Cloud Storage and Image upload - Instance snapshot via deltacloud core API
Date Mon, 12 Jul 2010 16:25:51 GMT
Task:
  "Implement Cloud Storage via the Deltacloud API where available, in
order to ultimately enable custom Image upload. In addition and where
image upload is not (yet) supported, implement 'Instance Snapshot'
functionality, where available. "

Hi All,

David, Michal and myself had a discussion last week about how we might
implement cloud storage via the deltacloud api with the ultimate goal of
allowing custom image upload to cloud storage. Only Amazon S3 and
Rackspace CloudFiles offer this type of generic 'upload an object' type
service (and only EC2 currently uses this storage for custom Images).

However, EC2,Rackspace and Gogrid do expose a 'snapshot a running server
and save as an Image' type function via their API, whilst Terremark and
Rimuhosting allow you to clone a running server but only to produce
another running server (i.e. cant save as an Image for later instantiation).

I think there are two independent subtasks here:

**Work out the operations for the generic cloud storage (i.e.
s3/cloudfiles) and implement 'Blob Management'. This will appear as a
top level collection (like Instances, Images etc, 'Blobs' or somesuch
name 'Objects'?) - to appear obviously only for those drivers that
support it, Amazon AWS and Rackspace.

**Implement the Instance snapshot function for those clouds where it
makes sense (e.g. do we want to implement the 'clone this Instance' type
functionality offered by Terremark and Rimuhosting?). This could be
exposed as an Instance 'action' (like stop, start, reboot) ?

I have just finished a review of the various APIs to see what each
exposes in terms of cloud storage and/or instance 'snapshotting'. I am
currently getting a list of operations together for the cloud storage
functions (s3/cloudfiles) and will start to implement this as an 
'Object-Store' top level collection.

The Instance snapshot function still needs some thought/discussion about
how/where we will implement this. I share my notes below for reference, 
please share your comments and thoughts on this if you are interested 
and get involved,

all the best, marios


========== NOTES ==========

=====================================================
Q. Which of our currently supported clouds offers a generic 'upload
data/blob/object to our storage service and access from anywhere even
from machines outside our cloud' - type storage?
=====================================================

--i--> Amazon Web Services (AWS) offers the Simple Storage Service (S3)
[1] - not to be confused with S4 [2]
--ii-> Rackspace offers CloudFiles [3]

     ** The services are very similar.
     ** Create 'buckets' in S3 and 'containers' is CloudFiles, and place
'objects' within these.
     ** Each S3 account is limited to 100 buckets, each holding an
unlimited number of objects with each at most 5 GBytes.
     ** A CloudFiles account can have unlimited buckets and objects;
again each object must be < 5 GBytes.

=====================================================
Q. Which of our currently supported clouds allows 'register custom Image
from cloud storage', or, 'save/snapshot an Instance as a new Image'
=====================================================

--i--> Amazon AWS allows you to register an object in S3 as an Image in
EC2 [4] (Rackspace 'coming soon' [5])

     ** The process of 'save this Instance as an Image in S3' is termed
'bundling'
     ** PROBLEM: different procedure for Windose vs Linux images [6].
	-- For Windows you use the AWS REST API 'BundleInstance' call where you 
specify the S3 bucket to upload to (amongst other things).
	-- For Linux you must install the 'AMI Tools' command-line utilities 
[7] on the Instance you wish to bundle and then 'ec2-bundle-image'. 
Confusingly the actual upload is done with 'ec2-upload-bundle' which 
takes the S3 bucket name as a parameter.

     ** In both cases and after bundling is complete you register the
newly created S3 object as an Image for EC2 using the REST API
'RegisterImage' call.

=====================================================

--ii-> Rackspace allows you to snapshot a running Instance [5]

     ** This is done through a REST API call (POST /images) where you
specify the serverid (Instance) that is to be saved.
     ** Here there is no seperation between 'make snapshot' and 'upload
to cloud storage' - i.e. cannot export a server image [8].
     ** Snapshots stay 'attached to the server' [9] (i.e. saved locally?).
     ** Once process complete the image is available for booting a server
[5].
     ** 'Custom Image Upload' is in the Cloud Server API Roadmap [5].

=====================================================

-iii-> Gogrid allows you to save a 'MyGSI - Gogrid Server Image' from
your own, preconfigured 'sandbox server'

     ** You create a MyGSI from a configured, running 'image sandbox
server' and the new image is added to Gogrid's cloud storage [11]
     ** Weirdly this cloud storage is not exposed at all via the API
     ** The API call is grid.image.save [12] - you create a new 'image
sandbox server' using 'grid.server.add', configure it and run the 'prep
scripts' and then call the save. Once complete the new MyGSI is
available in your list of images.

=====================================================

--iv-> Rimuhosting allows you to clone an existing server and deploy a
new server from that [10]

     ** However, the clone must be deployed straight away - you cannot
save as an image for later instantiation.
     ** A chat with support confirmed this - you *can* save an image for
later use but only by submitting a support ticket (i.e. nothing in the
api or the web 'management console' to support this).

=====================================================

--v--> Terremark VCloud Express allows you to 'clone' an existing server
(a 'vapp') [13] but you cannot save this as an Image (a 'vapp template').

     ** Similar to the rimuhosting situation - can clone an existing
server as another server, but not as an Image
     ** POST https://{Terremark URI}/vdc/{vDC ID}/action/clonevapp - you
specify the vApp id to be cloned within the request body.

=====================================================

AWS:
[1] http://aws.amazon.com/documentation/s3/
[2] http://www.supersimplestorageservice.com/
[4] http://docs.amazonwebservices.com/AWSEC2/latest/APIReference/
[6]
http://developer.amazonwebservices.com/connect/thread.jspa?messageID=107847&#107847
[7]
http://developer.amazonwebservices.com/connect/entry.jspa?externalID=368&categoryID=88
RACKSPACE:
[3] http://www.rackspacecloud.com/cloud_hosting_products/files/api
[5] http://docs.rackspacecloud.com/servers/api/cs-devguide-latest.pdf
[8] http://www.rackspacecloud.com/cloud_hosting_products/servers/faq
[9] http://www.rackspacecloud.com/cloud_hosting_products/servers/pricing
RIMUHOSTING:
[10]
http://apidocs.rimuhosting.com/jaxbdocs/com/rimuhosting/jaxrsorders/JAXRSOrders.NewVPSRequest.html
GOGRID:
[11] http://wiki.gogrid.com/wiki/index.php/Cloud_Storage:Cloud_Storage
[12] http://wiki.gogrid.com/wiki/index.php/API:grid.image.save
TERREMARK:
[13]
https://community.vcloudexpress.terremark.com/en-us/product_docs/w/wiki/6-vcloud-express-api-documention.aspx

Mime
View raw message