Return-Path: X-Original-To: apmail-deltacloud-commits-archive@www.apache.org Delivered-To: apmail-deltacloud-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 9655096B4 for ; Mon, 25 Jun 2012 12:34:23 +0000 (UTC) Received: (qmail 90722 invoked by uid 500); 25 Jun 2012 12:34:22 -0000 Delivered-To: apmail-deltacloud-commits-archive@deltacloud.apache.org Received: (qmail 90676 invoked by uid 500); 25 Jun 2012 12:34:22 -0000 Mailing-List: contact commits-help@deltacloud.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@deltacloud.apache.org Delivered-To: mailing list commits@deltacloud.apache.org Received: (qmail 89527 invoked by uid 99); 25 Jun 2012 12:34:16 -0000 Received: from tyr.zones.apache.org (HELO tyr.zones.apache.org) (140.211.11.114) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 25 Jun 2012 12:34:16 +0000 Received: by tyr.zones.apache.org (Postfix, from userid 65534) id 7A9076F5C; Mon, 25 Jun 2012 12:34:16 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: mfojtik@apache.org To: commits@deltacloud.apache.org X-Mailer: ASF-Git Admin Mailer Subject: [11/51] [partial] Site: Complete site revamp using bootstrap and docs update Message-Id: <20120625123416.7A9076F5C@tyr.zones.apache.org> Date: Mon, 25 Jun 2012 12:34:16 +0000 (UTC) http://git-wip-us.apache.org/repos/asf/deltacloud/blob/d3232da0/site/content/developers.mdown ---------------------------------------------------------------------- diff --git a/site/content/developers.mdown b/site/content/developers.mdown deleted file mode 100644 index 946ea47..0000000 --- a/site/content/developers.mdown +++ /dev/null @@ -1,557 +0,0 @@ ---- -title: Deltacloud - Information for Developers -extension: html -filter: - - markdown - - outline ---- -# Contribute to the Deltacloud Project - -[Apache](http://www.apache.org/ "The Apache Software Foundation") Deltacloud is -free, open source software licensed under the -[apache license](http://www.apache.org/licenses/LICENSE-2.0 "The Apache License"). -Please feel free to [contact us](contact.html) if you have any questions about the -project or with any suggestions on how to improve Deltacloud. The Deltacloud -developer community is open to anyone that wants to contribute! - -This page contains information for developers: - - - - - -
--------------------------------------- -
- -## Setting up the developer environment - -The information here is about setting up a dev environment for Deltacloud. If you -are interested in first trying out Deltacloud, you should instead follow the -instructions in the [Download](download.html) page and read the -[installation and quick start guide](documentation.html#install_deltacloud). - -Before setting up the development environment, make sure you have all the -[installation dependencies](documentation.html#dependencies). - -### Getting the sources - -Deltacloud provides the API server and drivers to various cloud providers. -It also includes a ruby client library and a commandline interface. - -**First, make sure you have installed all [dependencies](documentation.html#dependencies).** - -The Deltacloud repository is hosted at the Apache Software Foundation, -using the [git](http://git-scm.com/) version control system. The canonical -Deltacloud repository is at - with readonly -mirrors at `git://git.apache.org/deltacloud.git` and -`git://github.com/apache/deltacloud`. - -If you don't already have ***git*** there are -[git binaries](http://git-scm.com/download) available for specific systems. -Alternatively, if you use the ***yum*** or ***apt*** package managers: - - $ sudo yum install git - - $ sudo apt-get install git - -With ***git*** in place, to get the latest HEAD of the Deltacloud git repo: - - $ mkdir DeltacloudProject - $ cd DeltacloudProject - $ git clone git://git.apache.org/deltacloud.git - -
-[contents](#contents) - --------------------------------------- - -
- -### Development dependencies - -Apart from the [installation dependencies](documentation.html#dependencies), -in order to develop for Deltacloud you will need some additional libraries, -such as [cucumber](http://cukes.info/), -[ruby-debug](http://bashdb.sourceforge.net/ruby-debug/home-page.html) and -[rpsec](http://rspec.info/). The Deltacloud source includes a Gemfile which -lists these development dependencies: **/path/to/deltacloud/server/Gemfile**. - -Thus, the recommended way to get all development dependencies is with -[Bundler](http://gembundler.com/). If you don't already have bundler, install -it: - - $ sudo gem install bundler - -Then, you can get all the required developer dependencies by: - - $ cd /path/to/deltacloud/server - $ bundle install --system - - -
-[contents](#contents) - --------------------------------------- - -
- -### Building from source and installing the Deltacloud gem - -To build and install the Deltacloud server gem: - - $ cd path/to/DeltacloudProject/deltacloud/server - $ rake package - $ sudo gem install pkg/deltacloud-core-.gem - -Then install the Deltacloud client gem: - - $ cd path/to/DeltacloudProject/deltacloud/client - $ rake package - $ sudo gem install pkg/deltacloud-client-.gem - -
-[contents](#contents) - --------------------------------------- - -
- -## Contribute to Deltacloud - -The information here is aimed at helping developers more quickly understand the Deltacloud -project code, the process used by the development community for contributing and some -guidelines on how to go about adding a cloud provider driver to the Deltacloud project. - -### Overview of the Deltacloud directory structure - -**Note:** the following is not an exhaustive list of all files and directories in the -Deltacloud project. Rather it is intended as an aid to ease familirisation -with the project files for developers - for example to quickly identify where -the Deltacloud cloud provider drivers are stored: - - . - - deltacloud - |----------------------------------------------------------------------------------- - |-d-->tests Contains Cucumber tests - |----------------------------------------------------------------------------------- - |-d-->site Files for this website - |----------------------------------------------------------------------------------- - |-d-->client Contains the Deltacloud ruby client - |----------------------------------------------------------------------------------- - |-d-->clients Contains other Deltacloud clients (e.g. java) - |----------------------------------------------------------------------------------- - |-d--> server - |---------------------------------------------------------------------------- - |-f-->server.rb Contains the sinatra routes - |---------------------------------------------------------------------------- - |-d-->bin Contains the Deltacloud executable deltacloudd - |---------------------------------------------------------------------------- - |-d-->views Contains haml views for each collection - |---------------------------------------------------------------------------- - |-d-->tests Contains unit tests for drivers - |---------------------------------------------------------------------------- - |-d-->lib - |---------------------------------------------------------------------- - |-d-->sinatra Contains rabbit DSL and various helpers - |---------------------------------------------------------------------- - |-d-->deltacloud - |---------------------------------------------------------------- - |-d-->models Definition of each collection model - |---------------------------------------------------------------- - |-d-->drivers Contains the drivers for each cloud provider - |---------------------------------------------------------------- - |-d-->helpers Various helper methods used by the drivers - |---------------------------------------------------------------- - |-d-->base_driver Contains the Deltacloud base driver - |---------------------------------------------------------------- - -
-[contents](#contents) - --------------------------------------- - -
- -### Forming and sending patches - -There are many ways to work with git and there are many -[good resources](http://book.git-scm.com/) on using git. The Deltacloud community uses -a ***local_branch*** ---> ***patch*** ---> ***review*** ---> ***accept*** ---> ***commit*** -process for contributing to the project. The following shows the typical workflow used -by the core developers. - -First you need to set some settings inside the git configuration file. In the root -of the git project (Deltacloud in this case) there is a .git hidden directory containing -the config file. Open this with your preferred editor: - - $ vim deltacloud/.git/config - -Add the following to any existing configuration replacing the relevant values with your own: - - [sendemail] - signedoffbycc = no - chainreplyto = no - smtpserver = your_smpt.server.address - thread = yes - from = your_email_address - suppresscc = all - - [core] - whitespace = trailing-space,space-before-tab - - [apply] - whitespace = error-all - -After getting the latest HEAD of the Deltacloud git repo, you will have one local - branch called ***trunk***. - - $ git branch - * trunk - -Fetch the latest changes from the git repo: - - $ git pull - -Make a new local branch for your edits and give it a name: - - $ git checkout -b my_new_branch - -Make your edits/changes/voodoo. Once done, review which files you changed: - - $ git status - -Once you are happy with the changes and everything is working, commit those changes -to your local branch: - - $ git commit -a - -This will open an editor (e.g., vi) for you to enter a commit message. Please keep this -short and succint in general and keep individual lines at no longer than 80 characters. -The command **git log** will show you recent commits together with their commit messages -as examples. - -At this point, your local branch will contain the changes that you worked on and will -be different from your ***local*** **trunk** branch. In the meantime, other developers -may have already committed changes to the ***remote*** **trunk** branch in the Apache git repo. -Thus, you need to fetch any new changes and merge into your ***local*** **trunk** and then -rebase your local **trunk** onto the new branch which contains your changes. - -Change to trunk and fetch upstream changes if any: - - $ git checkout trunk - $ git pull - -Your ***local*** trunk is now up-to-date with the ***remote*** trunk in the git repo. -Next, rebase the ***local*** trunk onto the branch containing your changes: - - $ git rebase trunk my_new_branch - -Now you are ready to make patches against trunk and send them to the Deltacloud mailing -list for review by other developers in the community: - - $ git format-patch -o /path/to/where/you/keep/patches/ trunk - - $ git send-email --compose --subject 'some subject' - --thread /path/to/where/you/keep/patches/* --to dev@deltacloud.apache.org - -Your patches will be reviewed by other members of the community. The general rule used -is that a patch will be committed to the repo if it receives at least one **ACK** and no -**NACK**. Once a patch is **ACK**ed it will be committed by one of the Deltacloud community -developers that have commit rights to the Apache repo. If your patches are on the mailing -list for a few days and no-one is responding to them feel free to send a reminder email -as patches may be missed/forgotten. - -You can also contribute to the project by reviewing patches sent by other contributors: - -Make a new branch where you will apply the patches and test: - - $ git checkout -b jsmith_patches - -You can use either git am ("apply mail") to apply patches in mail format, or git apply for -plain-text patches. Using git am will also commit the patches to the local branch -preserving the author's commit messages whilst git apply will only apply the files but not -commit them. For simple patch review the difference between these is insignificant and -boils down to whether you like to save the patches as plain-text or in .mbox email format. - -Save the patches to some known location, make sure you're on the branch you wish to apply -them onto, and then apply: - - $ git checkout jsmith_patches - - $ cat /path/to/patches/0001-name-of-patch.txt | git apply - _OR_ - $ git am /path/to/patches/0001-name-of-patch.eml - -If you think the patches are sane, do what the author intended and most importantly don't -break anything else, then you can send an **ACK** in response to the patches on the -Deltacloud [mailing list](contact.html). Similarly, if you think there is some problem -with the patches you should send a **NACK** with an explanation of the problem you have -found so that the author can address this. - -
-[contents](#contents) - --------------------------------------- - -
- -### Use the Deltacloud github mirror - -The Deltacloud project maintains a -[github page](https://github.com/apache/deltacloud) which mirrors the Apache -repository. You can contribute through github if that is preferrable. -[Fork](http://help.github.com/fork-a-repo/) the Deltacloud repo, make your changes and -commit them back to your fork. Then issue a -[pull request](http://help.github.com/send-pull-requests/). It may also be wise to -send an email to the Deltacloud [mailing list](contact.html) to let the community know -about your pull request as in general the mailing list is checked much more often than -the github page. - -
-[contents](#contents) - --------------------------------------- - -
- -### Writing a new cloud provider driver - -The deltacloud drivers live in -[deltacloud/server/lib/deltacloud/drivers](#deltacloud_directories). The information here -is intended as a primer for anybody that wants to contribute a new cloud provider driver to -the Deltacloud project. - -To add a driver for a hypothetical **Foo** cloud, you should start by adding a a directory -under /drivers/ and then a file for the driver itself, i.e.: - - deltacloud/server/lib/deltacloud/drivers/foo/foo_driver.rb - -That file needs to define a class **Deltacloud::Drivers::Foo::FooDriver** and must be -a subclass of the Deltacloud::BaseDriver (example follows). - -You need to decide which [collections](api.html#h1) the provider for which you're -writing the driver supports - e.g. images, instances, keys, buckets/blobs (storage), etc. -and declare these with: - - def supported_collections - DEFAULT_COLLECTIONS + [ :buckets ] - [ :storage_snapshots, :storage_volumes ] - end - -The above declares that the driver supports the **DEFAULT_COLLECTIONS** (defined in -**deltacloud/server/lib/drivers.rb**) except [storage_snapshots](api.html#h4_2) and -[storage_volumes](api.html#h4_1) and additionally also supports the -[buckets](api.html#h4_3) collection. For example, a storage only cloud provider driver -would support only the **buckets** colletion. - -You can then start to define the methods for each of the collections that your driver -will support. The methods for each collection, as well as the routes that map to them -are defined in **/deltacloud/server/server.rb**. In general you can look at the existing -drivers for pointers on how to implement any specific method. - -One important consideration is how your cloud provider driver will be communicating -with the cloud provider. Many of the existing drivers use external ruby gems for this -purpose: for example, the **ec2** driver uses the [aws](https://github.com/appoxy/aws/) -gem, the **rackspace** driver uses the -[cloudfiles](https://github.com/rackspace/ruby-cloudfiles) and -[cloudservers](https://github.com/rackspace/ruby-cloudservers) gems. However, other drivers -implement their own clients with which to communicate with the cloud provider, such as -the IBM SBC driver and the Gogrid driver. This also explains why, under the /drivers -directory, some drivers contain only the provider\_***driver***.rb file, whilst others also -define a provider\_***client***.rb file. Whether you write your own client or use an existing -one is entirey at your discretion. - -Thus, your driver for the cloud provider foo may start out looking something like: - - require 'deltacloud/base_driver' - require 'foo_lib' # a library for talking to the foo cloud - - module Deltacloud - module Drivers - module Foo - - class FooDriver < Deltacloud::BaseDriver - - def supported_collections - DEFAULT_COLLECTIONS + [ :buckets ] - end - - def images(credentials, opts={}) - client = new_foo_client(credentials) - # use client to get a list of images from the back-end cloud and then create - # a Deltacloud Image object for each of these. Filter the result - # (eg specific image requested) and return to user - end - - def realms(credentials, opts={}) - (...) - end - - def instances(credentials, opts={}) - (...) - end - - ... ETC - - private - - def new_foo_client(credentials) - client = FooLib::Service.new({:user => credentials.user, - :pass => credentials.password }) - end - - end - end - end - end - -One important method for drivers that implement the [instances](api.html#h3_5) -collection is **instance_states**. This method represents the finite-state-machine -for instances which varies across cloud providers. For example, in some clouds -an instance may be in the 'running' state after creation, whereas in other clouds -an instance may need to be started explicitly. An example of what this method may -look like is: - - - define_instance_states do - start.to( :pending ) .on( :create ) - pending.to( :running ) .automatically - running.to( :running ) .on( :reboot ) - running.to( :stopping ) .on( :stop ) - stopping.to( :stopped ) .automatically - stopped.to( :finish ) .automatically - end - -The voodoo used here (i.e. definition of .to and .on etc) is defined in -/deltacloud/server/lib/deltacloud/state\_machine.rb. - -Valid states are - -- **`:begin`** -- **`:pending`** -- **`:running`** -- **`:stopping`** -- **`:stopped`** -- **`:end`** - -The `:begin` state is the state an instance is in immediate before -being created. The `:end` state is the state an instance is in -immediately after being destroyed. - -Valid transition actions are - -- **`:stop`** -- **`:start`** -- **`:reboot`** - -Additionally, to indicate a transition that may occur _without_ an -action being triggered, the action **`:automatically`** may be used, -as shown in the example above. - -Depending on the collections you'll be supporting, some other methods that -your driver may implement are: - -- `hardware_profiles(credentials, opts=nil)` -- `images(credentials, opts=nil )` -- `realms(credentials, opts=nil)` -- `instances(credentials, opts=nil)` -- `create_instance(credentials, image_id, opts)` -- `reboot_instance(credentials, id)` -- `stop_instance(credentials, id)` -- `destroy_instance(credentials, id)` - - -As a further example, the `hardware_profiles(...)` method should -return an array of `HardwareProfile` objects. The `opts` hash, if -present, must be inspected for `:id` and `:architecture` keys. If these keys -are present, the results should be filtered by the value associated with -each key. The `filter_on(...)` helper method is used for this filtering -and as you can see from existing driver method definitions, is invoked -in many of the driver collection methods: - - def hardware_profiles(credentials, opts=nil) - hardware_profiles = # get all hardware profiles from provider - hardware_profiles = filter_on( hardware_profiles, :id, opts ) - hardware_profiles = filter_on( hardware_profiles, :architecture, opts ) - return hardware_profiles - end - - -Once you've implemented some of the methods for your driver you can start to -'run' it and test that things are working as expected. To do this, you need to -create a yaml file for your driver so that the Deltacloud server 'knows' about -it. Assuming you've written the driver for cloud Foo and you've called it -foo\_driver.rb (which contains the class Deltacloud::Drivers::Foo::FooDriver) -, you need to drop a file into **/deltacloud/server/config/drivers** called -**foo.yaml**, containing the following: - - --- - :foo: - :name Foo - -You can then start the Deltacloud server with: - - deltacloudd -i foo - -
-[contents](#contents) - --------------------------------------- - -
- -### Writing and running tests - -If you add a new feature or write a new driver, you might like to add some tests to -make sure everything is running as expected. At present the Deltacloud project uses -Unit tests which live in **/deltacloud/server/tests** as well as Cucumber tests which live -in **/deltacloud/tests**. - -You can invoke the Unit tests: - - $ cd /path/to/deltacloud/server - $ rake test - -As you can see by inspecting the Rakefile in /deltacloud/server, this will invoke all -Unit tests defined in /deltacloud/server/tests. You can invoke specific driver tests by: - - $ cd /path/to/deltacloud/server - $ rake test:rackspace - _OR_ - $ rake test:mock - _etc_ - -You can invoke the Cucumber tests: - - $ cd /path/to/deltacloud/server - $ rake cucumber - -Alternatively, you can bypass the Rakefile and invoke the cucumber tests directly: - - $ cd /path/to/deltacloud/server - $ cucumber ../tests/mock - _OR_ - $ cucumber ../tests/ec2 - _etc_ - -
-[contents](#contents) - --------------------------------------- - -
- -### Licensing - the Apache Individual Contributor License Agreement - -Deltacloud is an [Apache Software Foundation](http://www.apache.org/) project. As such -and in keeping with all Apache projects, contributors are required to sign an individual -contributor license agreement. This, together with instructions on signing and sending -the agreement can be [found here](http://www.apache.org/licenses/icla.txt). Please do -not hesitate to [contact us](contact.html) if you have any queries or require any -clarification about contributing to the Deltacloud project. - -
-[contents](#contents) -
http://git-wip-us.apache.org/repos/asf/deltacloud/blob/d3232da0/site/content/doap_Deltacloud.rdf ---------------------------------------------------------------------- diff --git a/site/content/doap_Deltacloud.rdf b/site/content/doap_Deltacloud.rdf deleted file mode 100644 index cccc13b..0000000 --- a/site/content/doap_Deltacloud.rdf +++ /dev/null @@ -1,57 +0,0 @@ - - - - - - 2012-04-14 - - Apache DeltaCloud - - - Apache DeltaCloud software provides a universal API to abstract differences between cloud providers. - Apache DeltaCloud software allows you to start an instance on an internal cloud, then with the same code start another on EC2 or Rackspace. Deltacloud protects your apps from cloud API changes and incompatibilities, so you can concentrate on managing cloud instances the way you want. . - - - - Ruby - - - - Apache deltacloud 0.5 - 2012-02-03 - 0.5 - - - - - - - - - - - Apache DeltaCloud PMC - - - - - http://git-wip-us.apache.org/repos/asf/deltacloud/blob/d3232da0/site/content/documentation.haml ---------------------------------------------------------------------- diff --git a/site/content/documentation.haml b/site/content/documentation.haml deleted file mode 100644 index 3f8b5ed..0000000 --- a/site/content/documentation.haml +++ /dev/null @@ -1,558 +0,0 @@ ---- -title: Deltacloud - Documentation -area: documentation -extension: html -filter: - - haml ---- -%h1 Documentation - -%p The Deltacloud documentation is divided into the following parts: - -%ul - - documentation_pages.each do |doc| - - if doc[:description] - %li - %a{ :href => doc[:href] } - = doc[:description] - -%br -%br -%hr -%br - -%a(name="install_deltacloud") -%h1 Install Deltacloud - -This page lists the libraries and packages that you will need in order to install -the Deltacloud server, a "Hello Deltacloud" quick-start guide as well as some -information on interacting with the Deltacloud server after a succesful installation. - -%h2 Installation of Deltacloud itself - -Once you've setup all the dependencies listed below, installing Deltacloud is as easy as -typing: -%br -%br -%pre - %code - $ sudo gem install deltacloud-core -And -%strong thats it! -The -%strong gem install -command will automatically fetch and install all other gems that the Deltacloud server -needs if you don't already have these. As an alternative you can get the latest -releases of Deltacloud from the -%a{:href=>"http://www.apache.org/dist/deltacloud/"} - Apache website. - -%br -%br -%hr -%br - -%a(name="dependencies") - -%h2 Installation dependencies - -The Deltacloud server relies on a number of external rubygems and other libraries. -You must have the following packages installed before trying to install and -run the Deltacloud server. Please note that the names of the packages below -may differ for your distribution (e.g. 'ruby-devel' vs 'ruby-dev'). In short, you need: -%br -%br -%ul - %li ruby and ruby-devel, - %li gem (RubyGems), - %li gcc-c++, - %li libxml2 and libxml2-devel, - %li libxslt and libxslt-devel, - %li rake - -More info on getting these follows: - -%h3 ruby and ruby-devel - -You need to -%a{ :href=> "http://www.ruby-lang.org/en/downloads/"} - install ruby. -You can check to see if you already have a ruby installation by typing: -%br -%br -%pre - %code - $ ruby -v - -If you see something that looks like the following: -%br -%br -%pre - %code - ruby 1.8.7 (2010-08-16 patchlevel 302) [i686-linux] - -then you can skip the ruby installation. Deltacloud requires ruby to be -at least version 1.8.7. You also need the development headers (ruby-devel) as -Deltacloud relies on some rubygems with C extensions. If you use a package manager -such as yum or apt-get: - -%br -%br -%pre - %code - sudo yum install ruby - sudo yum install ruby-devel -OR -%br -%br -%pre - %code - sudo apt-get install ruby - sudo apt-get install ruby-devel - -%h3 gem (RubyGems) - -Deltacloud relies on a number of ruby gems so you also need to -%a{:href=> "http://docs.rubygems.org/read/chapter/3"} - install RubyGems. -As with ruby, you can check if you already have the -%strong gem -executable installed by typing -%strong gem -v -\. Again, your package manager can help with the installation: -%br -%br -%pre - %code - $ sudo yum install rubygems - - -%h3 gcc-c++, libxml2, libxml2-devel, libxslt, libxslt-devel - -These are required to build RubyGems that have C extensions. Again, typically -the easiest way to get these is with you package manager: - -%br -%br -%pre - %code - $ sudo yum install gcc-c++ - $ sudo yum install libxml libxml2-devel - $ sudo yum install libxslt libxslt-devel - -%h3 rake -%a{:href=>"http://rake.rubyforge.org/"} - Rake -is Ruby's Make and is itself a ruby gem. Once you have RubyGems installed you can -get rake with: -%br -%br -%pre - %code - $ sudo gem install rake - -%br -%br -%hr -%br - -%a(name="quick_start") -%h1 "Hello Deltacloud!", quick-start guide - -This guide assumes a linux environment. Some of the Deltacloud developers -are running recent versions of -%a{:href=> "http://fedoraproject.org"} - Fedora. -However, as Deltacloud is written in Ruby you should be able to install it on -any OS that supports Ruby. If you have succesfully installed Deltacloud or are -having any problems doing so, please -%a{:href=>"contact.html"} - let us know, -we'd love to hear from you! - -%h2 Running the Deltacloud server - -After you succesfully install Deltacloud, you can start the server by typing -%strong - deltacloudd -i - %em - provider_id. -The -%strong - %em - provider_id -is the name of the cloud provider that you have an account with, for example 'ec2'. -If you don't yet have an account with a cloud provider, you can still try the Deltacloud -mock driver: - -%br -%br -%pre - %code - deltacloudd -i mock - - -This will start the Deltacloud server on your local machine with the mock driver, accepting -connections on port 3001 (the default). From version 0.4.0 of Deltacloud, you can use the -'-l' flag to see -%a{:href=> "drivers.html#providers"} - all available -%strong - %em - provider_ids -that can be used with the -%strong deltacloudd -executable. - -%br -%br -%pre - %code - $ deltacloudd -l - %br - Available drivers: - * condor - * vsphere - * opennebula - * eucalyptus - * rhevm - * sbc - * azure - * gogrid - * mock - * rackspace - * rimuhosting - * terremark - * ec2 - -After you start the server, the Deltacloud HTML interface is available at -%strong - http://localhost:3001/api -\- open this address in your web browser. If you want to see XML output from the -server in the browser, append -%strong - %em - format=xml -to each URL. On Webkit based browsers like Safari, you might need to instruct the -server explicitly to -%a{ :href => "http://www.gethifi.com/blog/webkit-team-admits-accept-header-error" } - return HTML -Do this by appending -%strong - %em - format=html -to each URL. Your browser will prompt you for -%a{:href=> "drivers.html#credentials"} - credentials -when you invoke an operation that requires -%a{:href=> "api.html#authentication"} - authentication. - - -%br -%br - -The '-h' flag will list all available options for -%strong deltacloudd. -For example, to start the Deltacloud server with the Rackspace driver on port 10000 -you can use: - -%br -%br -%pre - %code - $ deltacloudd -i rackspace -P 10000 - -You may want to install the server on another machine and make Deltacloud available on -your local network. To do this you need to bind the Deltacloud server to an address other -than 'localhost' (default). For instance, if you install and start Deltacloud on a machine -with the IP address 192.168.10.200 you should start the server with: - -%br -%br -%pre - %code - $ deltacloudd -i ec2 -P 5000 -r 192.168.10.200 - -This will make the Deltacloud server available at the address -%strong - http://192.168.10.200:5000/api. -%br -%br - - -%br -%br -%hr -%br - -%h2 The Deltacloud ruby client - -The Deltacloud project also maintains a ruby client, which you need to install -seperately to the Deltacloud API server. The Deltacloud client consists of a -ruby library (packaged as a ruby gem) which you can use to programmatically -interact with the Deltacloud server and control your cloud infrastructure -across cloud providers. For example, assuming the Deltacloud server is -running at - -%strong - 192.168.10.244:5000 - -%br -%br -%pre - %code - require 'deltacloud' - %br - api_url = 'http://192.168.10.244:5000/api' - api_name = 'TK2PJCAN9R1HKG2FK24Z' - api_password = 'aLe27rZlRhlBcVoQbL4JsVtaNga12vEL9d9kS5CA' - %br - client = DeltaCloud.new( api_name, api_password, api_url ) - %br - \# get a list of currently running instances (virtual machines) - client.instances.each do |instance| - \ puts instance.name - end - %br - #do other work with client here - -More comprehensive information on using the Deltacloud ruby client in this way -can be found in the -%a{:href=>"client-ruby.html"} - client documentation. - -Assuming you already have Ruby and RubyGems setup, you can install the Deltacloud -client by simply typing: - -%br -%br -%pre - %code - $ sudo gem install deltacloud-client - -Installing the Deltacloud client also gives you the -%strong - deltacloudc -command line tool. This executable makes use of the Deltacloud client library to speak to the -Deltacloud server using the -%a{:href=>"api.html"} - REST API. -This allows you to to control your IAAS cloud infrastructure form the command line; -even better if you are familiar with linux scripting, you can knock up a bash script in no time -to automate your cloud infrastructure tasks. - -The general usage pattern for deltacloudc is: - -%br -%br -%pre - %code - $ deltacloudc collection operation [options] - -%ul - %li - %strong - %em - collection - refers to the Deltacloud object collections, such as Instances, - Images, Buckets, Realms etc, as described in greater detail in the - %a{:href=> "api.html#h1"} - REST API - %li - %strong - %em - operation - is collection dependant. All collections respond to 'index' - and 'show' operations (retrieve details on all objects in a given collection - or on a specific object, respectively); some collections respond to 'create' and 'destroy' - operations. The instances collection (realised virtual servers) responds to operations - for managing the instance lifecycle, such as 'stop', 'reboot' etc. - %li - %strong - %em - options - are listed by invoking - %strong deltacloudc -h - One important option is - %strong -u, - with which you specify the API_URL where the Deltacloud server is running. The API_URL - takes the form - %strong - http://[user]:[password]@[api_url]:[port]/[api] - (examples follow). Alternatively, rather than having to supply the API_URL for every - invocation of - %strong - deltacloudc - you have the choice of setting the API_URL environment variable - (e.g., export API_URL=http://mockuser:mockpassword@localhost:3001/api). A - listing of the credentials you need to provide for each back-end cloud provider - is available - %a{:href=>"drivers.html#credentials"} - here. - -The following examples assume that the Deltacloud server is running on your local machine -port 3001 (the -%strong deltacloudd -server daemon defaults to 'localhost:3001') and was started with the 'mock' provider -(i.e. -%strong deltacloudd -i mock -). - -To list all collections available in the current driver: -%br -%br -%pre - %code - $ deltacloudc -l -u http://mockuser:mockpassword@localhost:3001/api - -To get a list of all -%strong images: -%br -%br -%pre - %code - $ deltacloudc images index -u http://mockuser:mockpassword@localhost:3001/api - -To list all operations for the -%strong buckets -collection: -%br -%br -%pre - %code - - $ deltacloudc buckets -l -u http://mockuser:mockpassword@localhost:3001/api - -To create a new blob in the bucket called 'bucket1': -%br -%br -%pre - %code - - $ deltacloudc blob create -i 'my_new_blob' -b 'bucket1' -f /home/marios/file.txt - - -%br -%br -%hr -%br - - -%h2 Other HTTP clients - cURL - -As interaction with the Deltacloud server is via HTTP calls, you can use any -HTTP client to talk to it using the Deltacloud -%a{:href=>"api.html"} - REST API. -One popular command line tool available on most modern linux distributions is -%a{:href=> "http://curl.haxx.se/"} - cURL. -We give some examples here of how you can use cURL to interact with Deltacloud, -demonstrating the flexibility and power of the Deltacloud REST API. - -The following examples assume the Deltacloud server is running on locahost:3001, -and was started with the 'ec2' driver (i.e., -%strong deltacloudd -i ec2 -). - -To get a -%a{:href=>"api.html#list_images"} - listing of all images -available in the back-end cloud: - -%br -%br -%pre - %code - curl --user "pGbAJ1TsVg5PKs3BK27O:dPs47ralgBlldqYNbLg3scthsg4g8v0L9d6Mb5DK" - "http://localhost:3001/api/images?format=xml" - -The cURL -%strong --user -option is used to specify the -%strong username:password -%a{:href=>"drivers.html#credentials"} - credentials -for access to the back-end cloud provider (Amazon EC2 in this case). - -To -%a{:href=>"api.html#create_instance"} - create a new instance -from the -%a{:href=>"api.html#h3_3"} - image -with id 'ami-f51aff9c', in -%a{:href=>"api.html#h3_1"} - realm -'us-east-1c', with the -%a{:href=>"api.html#h3_2"} - hardware profile -'c1.medium', in -%a{:href=>"api.html#h3_7"} - firewall -'default': - -%br -%br -%pre - %code - curl -X POST -F "keyname=eftah" -F "image_id=ami-f51aff9c" - \-F "realm_id=us-east-1c" -F "hwp_id=c1.medium" -F "firewalls1=default" - \--user "pGbAJ1TsVg5PKs3BK27O:dPs47ralgBlldqYNbLg3scthsg4g8v0L9d6Mb5DK" - "http://localhost:3001/api/instances?format=xml" - -To -%a{:href=>"api.html#delete_firewall"} - delete a firewall -called 'develgroup': -%br -%br -%pre - %code - curl -X DELETE - \--user "pGbAJ1TsVg5PKs3BK27O:dPs47ralgBlldqYNbLg3scthsg4g8v0L9d6Mb5DK" - http://localhost:3001/api/firewalls/develgroup?format=xml - -To -%a{:href=>"api.html#create_blob"} - create a blob -called 'my_new_blob' within the -%a{:href=>"api.html#h4_3"} - bucket -'mybucket' from a local file with -%strong - %em - HTTP PUT -specifying its content type and setting some some metadata -%strong - %em - key:value -pairs: - -%br -%br -%pre - %code - curl -H 'content-type: text/html' -H 'X-Deltacloud-Blobmeta-Name:mariosblob' - \-H 'X-Deltacloud-Blobmeta-Version:2.1' --upload-file - "/home/marios/Desktop/somefile.html" - \--user "pGbAJ1TsVg5PKs3BK27O:dPs47ralgBlldqYNbLg3scthsg4g8v0L9d6Mb5DK" - http://localhost:3001/api/buckets/mybucket/my_new_blob?format=xml - -To -%a{:href=>"api.html#h4_3_10"} - retrieve blob metadata -for the blob called 'my_new_blob': - -%br -%br -%pre - %code - curl -iv -X HEAD - \--user "pGbAJ1TsVg5PKs3BK27O:dPs47ralgBlldqYNbLg3scthsg4g8v0L9d6Mb5DK" - http://localhost:3001/api/buckets/mybucket/my_new_blob?format=xml - -Note the use of the '-iv' flags here which will ensure that cURL displays the -request and response headers (blob metadata is reported in the response headers, -with an empty response body). http://git-wip-us.apache.org/repos/asf/deltacloud/blob/d3232da0/site/content/download.mdown ---------------------------------------------------------------------- diff --git a/site/content/download.mdown b/site/content/download.mdown deleted file mode 100644 index 9da2635..0000000 --- a/site/content/download.mdown +++ /dev/null @@ -1,102 +0,0 @@ ---- -title: Deltacloud - Download -extension: html -filter: - - markdown ---- - -Deltacloud conceptual overview - -Deltacloud provides the API server and drivers necessary for connecting -to the [cloud providers](./drivers.html#providers "Deltacloud drivers"). -Install Deltacloud and start the ***deltacloudd*** daemon server. You can use your -favourite HTTP client to talk to the server using the -[Deltacloud REST API](api.html "Deltacloud REST API"). Deltacloud even comes with a -HTML interface so you can simply use your web browser to control your cloud -infrastructure straight out of the box. As of Deltacloud 0.4.0, the HTML interface -is written with the [jQuery mobile](http://jquerymobile.com "jquery mobile") -framework so it plays especially well with your mobile or tablet devices. -
-This page lists the location of the main 'bits' of Deltacloud - for more detailed -information on getting and installing Deltacloud checkout out our -[installation and quick start guide](documentation.html#install_deltacloud). -
--------------------------------------- -
- -# Get Deltacloud # - -Deltacloud is available as a ruby gem. First make sure you have installed all the -[required dependencies](documentation.html#dependencies), -Then installing Deltacloud is as easy as typing: - - $ sudo gem install deltacloud-core - -**And thats it!** Yes, really. The **gem install** command will automatically fetch and install -all other gems that the Deltacloud server needs if you don't already have these. - -As an alternative you can get the latest releases of Deltacloud from the -[Apache website](http://www.apache.org/dist/deltacloud/). - -## Getting the sources ## - -More information for developers is available [here](developers.html). - -The Apache Deltacloud git repo is at: -**https://git-wip-us.apache.org/repos/asf/deltacloud.git**. - - git clone https://git-wip-us.apache.org/repos/asf/deltacloud.git - -You can also use the readonly git mirrors at Apache and -[Github](https://github.com/apache/deltacloud): - - git clone git://git.apache.org/deltacloud.git - git clone git://github.com/apache/deltacloud - -
-
--------------------------------------- -
- -# The Deltacloud Ruby Client # - -The Deltacloud project also maintains a ruby client, which you need to install -separately to the Deltacloud API server. The Deltacloud client consists of a - ruby library (packaged as a ruby gem) which you can use to programmatically -interact with the Deltacloud server and control your cloud infrastructure -across cloud providers. - -Assuming you already setup the [required dependencies](documentation.html#dependencies), -you can install the Deltacloud by simply typing: - - $ sudo gem install deltacloud-client - -Installing the Deltacloud client also gives you the **deltacloudc** command line -tool. This executable makes use of the Deltacloud client library to speak to the -Deltacloud server using the [REST API](api.html). More information on getting started -with the Deltacloud Ruby client is available in the -[quick start guide](documentation.html#quick_start). - -
-
--------------------------------------- -
- -# libdeltacloud # - -A library for accessing the Deltacloud API from C/C++ programs. - -You can get the source code: - - $ git clone git://git.fedorahosted.org/deltacloud/libdeltacloud.git - -Or you can download the RPMs for Fedora 12 and 13: - - - -Contributed by Chris Lalancette. - -# Your name here # - -Do you have a tool to make Deltacloud more useful? [Let us know](contact.html) -and we'll add your contributions to this page! http://git-wip-us.apache.org/repos/asf/deltacloud/blob/d3232da0/site/content/drivers.haml ---------------------------------------------------------------------- diff --git a/site/content/drivers.haml b/site/content/drivers.haml deleted file mode 100644 index 690aff6..0000000 --- a/site/content/drivers.haml +++ /dev/null @@ -1,17 +0,0 @@ ---- -title: Deltacloud - Documentation -area: documentation -extension: html -filter: - - haml ---- -- page = render(:partial => "external", :locals => { :path => 'content/_drivers.mdown' }) -- header, rest = page.split("
") - -= header - -= render(:partial => "providers") - -= render(:partial => "credentials") - -= rest http://git-wip-us.apache.org/repos/asf/deltacloud/blob/d3232da0/site/content/drivers.md ---------------------------------------------------------------------- diff --git a/site/content/drivers.md b/site/content/drivers.md new file mode 100644 index 0000000..873ac58 --- /dev/null +++ b/site/content/drivers.md @@ -0,0 +1,712 @@ +--- +site_name: Deltacloud API +title: Drivers +--- + +
+ +
+
+ +

Deltacloud drivers

+ +

+Deltacloud provides drivers for a growing number of popular IaaS cloud providers. This page contains notes relevant to specific cloud provider drivers, such as the credentials that should be used for a given cloud provider. Furthermore the information here outlines the mechanism through which any API call sent by a given client to a Deltacloud server instance may be routed to a specific driver, regardless of the 'default' driver that the Deltacloud server was invoked for. +

+ +

Dynamic driver switching

+ +

+When the Deltacloud server is started it is passed a parameter that specifies the default driver to be used for API operations: +

+ + +
+
+ + + +
+
+ +
+$ deltacloudd -i ec2
+
+ +

+The above example shows how to start the Detlacloud server with the Amazon EC2 driver. It is possible to start a number of Deltacloud server instances for each cloud provider that you wish to connect to (e.g. on different ports). There is also a mechanism is with which you can instruct the server to use a specific driver, regardless of the current default. The Deltacloud API drivers collection (e.g. GET /api/drivers) provides a list of all currently supported cloud provider drivers. +

+ +

+Some drivers also support the notion of a provider. Changing the provider makes it possible to use the same driver against different instances of a cloud, for example different regions in EC2 or different installations of RHEV-M. The possible range of values for the provider is driver-specific, and it is listed in the notes for each driver below. +

+ +

+The driver and provider can be selected through the request headers X-Deltacloud-Driver and X-Deltacloud-Provider. For example, including the headers X-Deltacloud-Driver: ec2 and X-Deltacloud-Provider: eu-west-1 ensures that a request will be serviced by the EC2 driver, and that the driver will use the eu-west-1 region in EC2. +

+ +

Driver functionality and Credentials

+ +

Compute Driver Functionality

+ + + + + + + + + + + + + + + + + + + + + + + + + + + +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  + Create new instances + + Start stopped instances + + Stop running instances + + Reboot running instances + + Destroy instances + + List all/get details about hardware profiles + + List all/get details about realms + + List all/get details about images + + List all/get details about instances +
+ Amazon EC2 + yesnoyesyesyesyesyesyesyes
+ Eucalyptus + yesnoyesyesyesyesyesyesyes
+ Fujitsu FGCP + yesyesyesnoyesyesyesyesyes
+ IBM SBC + yesyesyesyesyesyesyesyesyes
+ GoGrid + yesnoyesyesyesyesyesyesyes
+ OpenNebula + yesyesyesyesyesyesyesyesyes
+ Rackspace + yesnoyesyesyesyesyesyesyes
+ RHEV-M + yesyesyesnoyesyesyesyesyes
+ RimuHosting + yesyesyesyesyesyesyesyesyes
+ Terremark + yesyesyesyesyesyesyesyesyes
+ vSphere + yesyesyesyesyesyesyesyesyes
+ OpenStack + yesyesyesyesyesyesyesyesyes
+ +
+ +

Storage Driver Functionality

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
  + Create new buckets + + Update/delete buckets + + Create new blobs + + Update/delete blobs + + Read/write blob attributes + + Read/write individual blob attributes +
+ Amazon S3 + yesyesyesyesyesyes
+ Eucalyptus Walrus + yesyesyesyesyesyes
+ Rackspace CloudFiles + yesyesyesyesyesyes
+ Microsoft Azure + yesyesyesyesyesyes
+ Google Storage + yesyesyesyesyesyes
+
+

+Deltacloud uses basic HTTP authentication to receive credentials from the client and passes them through to the particular back-end cloud. The credentials always consist of a username and password and they are never stored on the server. The exact credentials for logging into the server, and a place where you can find them, depends on the backend cloud that the server is talking to. +

+ +

+The following table gives details about the credentials that must be provided for each of the supported clouds. The entry from the Driver column needs to be passed as the -i option to the deltacloudd server daemon. Note that some of the drivers require additional information, e.g. API endpoint URL's. For more details see the Notes on specific drivers section. +

+ + + +

Cloud provider credentials

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
CloudDriverUsernamePasswordNotes
+ mock + mockmockusermockpasswordThe mock driver does not communicate with any cloud; it just pretends to be a cloud.
+ Amazon EC2/S3 + ec2Access Key IDSecret Access KeyRetrieve neccessary information from the Security Credentials page in your AWS account.
+ Eucalyptus + eucalyptusAccess Key IDSecret Access Key
+ Fujitsu FGCP + fgcpUser certificate's folder nameUser certificate's passphrase +

Set the environment variable FGCP_CERT_DIR to a folder where the folder with UserCert.p12 is stored or place UserCert.p12 in

+

+~/.deltacloud/config/fgcp/<Username>/ +  or

+

+%USERPROFILE%\.deltacloud\config\fgcp\<Username>\ +

+

in Windows. Then use Username to authenticate.

+
+ GoGrid + gogridAPI KeyShared Secret + Go to My Account > API Keys for your account and click on the key you want to use to find the Shared Secret. +
+ IBM SBC + sbcUsernamePassword
+ Microsoft Azure (Storage Account only) + azurePublic Storage Account NamePrimary Access Key + The Storage Account Name is chosen when you create the service (e.g. name in name.blob.core.windows.net). The name and the access key are available from the service control panel. +
+ OpenNebula + opennebulaOpenNebula userOpenNebula passwordSet the environment variable OCCI_URL to the address on which OpenNebula's OCCI server is listening.
+ OpenStack + openstackOpenStack userOpenStack user passwordSet the environment variable API_PROVIDER to the URL of OpenStack API entrypoint.
+ Rackspace Cloud Servers/Cloud Files + rackspaceRackspace user nameAPI KeyObtain the key from the API Access page in your control panel.
+ RHEV-M + rhevm + RHEV-M user name plus Windows domain, e.g., admin@rhevm.example.com + RHEV-M passwordSet the environment variable API_PROVIDER to the URL of the RHEV-M REST API endpoint.
+ Rimuhosting + rimuhostingnot used (?)API Key
+ Terremark + terremarkUsernamePassword
+ VMware vSphere + vspherevSphere uservSphere user passwordSet the environment variable API_PROVIDER to the hostname of the vSphere server.
+ +
+ +

Notes on specific drivers

+ +
+
+
+ + EC2 driver + +
+
+
+ The providers for the EC2 driver correspond to AWS's regions, and currently are us-east-1, us-east-1, eu-west-1, ap-southeast-1. +
+
+
+
+
+ + Eucalyptus driver + +
+
+
+

+The Eucalyptus driver is based on the EC2 driver. +

+ +

+The driver allows selecting the Eucalyptus installation by setting a provider in the format +

+ +

+For example, for the Eucalyptus installation at 192.168.1.1:8773 and a Walrus installation at 192.168.1.2:8773, the driver can be pointed at that installation by passing the request headers +

+ +
+X-Deltacloud-Driver: eucalyptus
+X-Deltacloud-Provider: ec2=192.168.1.1:8773;s3=192.168.1.2:8773
+
+
+
+
+
+
+ + IBM Smartcloud driver + +
+
+
+ When using the IBM SmartCloud driver, the credentials passed in response to the HTTP 401 authentication challenge should be your IBM SmartCloud username and password. +
+
+
+
+
+ + OpenNebula + +
+
+
+

+When you use the OpenNebula driver, the credentials passed in response to the HTTP 401 authentication challenge should be your OpenNebula user and password. +

+ +

+The address, on which the OCCI server is listening, needs to be defined in an environment variable called OCCI_URL. +

+ +

+The OpenNebula driver has been updated to support v3.x of the OpenNebula API. The driver is contributed by Daniel Molina who has also put together a guide for using OpenNebula through Deltacloud. +

+
+
+
+
+
+ + Fujitsu FGCP driver + +
+
+
+

+ When you use the Fujitsu FGCP driver, do not authenticate with your FGCP Portal username. Use the name of the folder in which your UserCert.p12 is stored. UserCert.p12 can be issued from the Portal and it is the same as you use to access MyPortal. +

+

+ Set the enviroment variable FGCP_CERT_DIR to override the default path ~/.deltacloud/config/fgcp/ to locate the Username folder with the UserCert.p12 file. +

+
+
+
+
+
+ + OpenStack driver + +
+
+
+ To connect to OpenStack API, you will need to set the API_provider environment variable or the 'X-Deltacloud-Provider' HTTP header to a valid OpenStack API entrypoint. +
+
+
+
+
+ + Rackspace driver + +
+
+
+ When you use the Rackspace-cloud driver (Rackspace cloud used to be called "Mosso") - the password in a HTTP 401 challenge should be your API key, NOT your Rackspace account password. You can get the API-key, or generate a new one, from the Rackspace console. +
+
+
+
+
+ + RHEV-M driver + +
+
+
+

+The RHEV-M driver supports latest release of Red Hat Enterprise Virtualization Manager (3.0 currently). In order to make the driver work with this provider, you need to set the API_PROVIDER environment variable or use the 'X-Deltacloud-Provider' request header to the URL of the RHEV-M REST API entry point. The usual URL looks like: +

+ +
+API_PROVIDER="https://rhevm.hostname.com:8443/api;645e425e-66fe-4ac9-8874-537bd10ef08d" 
+
+ +

+To make sure that you have right credentials, try to access https://rhevm.hostname.com:8443/rhevm-api in your browser. If you're able to authenticate within the browser, then the crendentials you used are valid Deltacloud credentials. +

+ +

+In order to make RHEV-M driver work properly, you need to set the RHEV-M Data Center UUID you want to speak with in API_PROVIDER url (see the example above). To obtain a list of all Data Centers you can choose from, start Deltacloud API without specifying a datacenter in the API_PROVIDER URL and do this request: +

+ +
+GET /api/drivers/rhevm?format=xml
+
+ +

+The list of possible datacenters will appear in the 'providers' section. +

+
+
+
+
+
+ + RimuHosting + +
+
+
+ Further details coming soon. +
+
+
+
+
+ + VMware vSphere driver + +
+
+
+

+You can find the details on how to make the VMware vSphere driver work with Deltacloud API in vSphere Setup in Aeolus project wiki. +

+ +

+In order to connect to vSphere, you need to set the API_PROVIDER environment variable or use the 'X-Deltacloud-Provider' HTTP header in the request to the vSphere host you want to use. For example: +

+ +
API_PROVIDER="vsphere.hostname.com"
+ +

+The username and password in 401 challenge should be the same as you use in the vSphere Control Center. +

+
+
+
+
+ http://git-wip-us.apache.org/repos/asf/deltacloud/blob/d3232da0/site/content/firewalls.md ---------------------------------------------------------------------- diff --git a/site/content/firewalls.md b/site/content/firewalls.md new file mode 100644 index 0000000..a4036c1 --- /dev/null +++ b/site/content/firewalls.md @@ -0,0 +1,442 @@ +--- +site_name: Deltacloud API +title: Firewalls +--- + +
+ +
+
+ +

Firewalls

+ +

+Firewalls represent sets of rules that govern the accessibility of a running instance over the public Internet. At present, only the Amazon EC2 cloud supports this collection (Amazon EC2 'Security Groups'). A firewall has these attributes: +

+ +
    +
  • a name
    • +
  • a description
    • +
  • an owner_id
    • +
  • set of rules
    • +
+ +

+An instance is launched into a firewall by specifying the firewalls1 ... firewallsN parameters in the POST /api/instances operation (see the Create an instance section). +

+ +

+Each firewall rule has a number of attributes describing the access granted to clients that want to communicate with the instance over the Internet. Each rule cosists of +

+ +
    +
  • an allow_protocol (tcp, udp or icmp);
    • +
  • a port_from and a port_to that delimit the port range for access; and
    • +
  • a sources list, which can contain firewalls (i.e. allow instances in another firewall to communicate with instances in the firewall in which this rule exists), or a number of IP addresses in CIDR format, or a mix of both.
    • +
+ +
+
+ + + +
+
+ +

+Each rule also specifies a direction, indicating whether it applies to ingress or egress traffic. +

+ +

+As Amazon EC2 has no notion of a firewall rule ID, the Deltacloud server constructs one ID for each rule as the concatenation of its attributes. The format used is: owner_id~protocol~from_port~to_port~@sources. +

+ +

+As explained above a source can be the address type in which case it defines an IP type (ipv4/ipv6), an IP address and routing prefix (CIDR netmask). Sources of type group have an owner_id and a name. The owner_id is the identifier of the account that created the specified firewall. The name defines the firewall to which the access is being granted. +

+ +

+An example of a rule id is: +

+ +
+393485797142~tcp~22~22~@group,393485797142,default,@address,ipv4,10.1.2.3,24
+          {owner_id~protocol~from_port~to_port~@sources}
+
+ +

+By creating the rule identifier abstraction, the Deltacloud API supports deletion of an entire firewall rule as one operation. +

+ +
+ + + +
+ +
+
+ +

Get a list of all firewalls

+ +

+To retrieve a list of all firewalls use call GET /api/firewalls. +

+ +

Example request:

+ +
+GET /api/firewalls?format=xml HTTP/1.1
+Authorization: Basic AU1J3UB2121Afd1DdyQWxLaTYTmJMNF4zTXBoRGdhMDh2RUw5ZDAN9zVXVa==
+User-Agent: curl/7.20.1 (i386-redhat-linux-gnu)
+Host: localhost:3001
+Accept: */*
+
+ +

Server response:

+ +
+HTTP/1.1 200 OK
+Content-Type: application/xml
+Date: Tue, 26 Jul 2011 15:56:04 GMT
+Content-Length: 1640
+
+<?xml version='1.0' encoding='utf-8' ?>
+<firewalls>
+  <firewall href='http://localhost:3001/api/firewalls/default' id='default'>
+    <name><![CDATA[default]]></name>
+    <description><![CDATA[default group]]></description>
+    <owner_id>393485797142</owner_id>
+    <rules>
+      <rule id='393485797142~tcp~22~22~@address,ipv4,87.228.192.251,32'>
+        <allow_protocol>tcp</allow_protocol>
+        <port_from>22</port_from>
+        <port_to>22</port_to>
+        <direction>ingress</direction>
+        <sources>
+          <source address='87.228.192.251' family='ipv4' prefix='32' type='address'></source>
+        </sources>
+      </rule>
+    </rules>
+  </firewall>
+  <firewall href='http://localhost:3001/api/firewalls/test' id='test'>
+    <name><![CDATA[test]]></name>
+    <description><![CDATA[this is just a test]]></description>
+    <owner_id>393485797142</owner_id>
+    <rules>
+      <rule id='393485797142~tcp~22~22~@group,393485797142,default,@address,ipv4,10.1.2.3,24'>
+        <allow_protocol>tcp</allow_protocol>
+        <port_from>22</port_from>
+        <port_to>22</port_to>
+        <direction>ingress</direction>
+        <sources>
+          <source name='default' owner='393485797142' type='group'></source>
+          <source address='10.1.2.3' family='ipv4' prefix='24' type='address'></source>
+        </sources>
+      </rule>
+    </rules>
+  </firewall>
+  <firewall href='http://localhost:3001/api/firewalls/new_firewall' id='new_firewall'>
+    <name><![CDATA[new_firewall]]></name>
+    <description><![CDATA[new_one]]></description>
+    <owner_id>393485797142</owner_id>
+    <rules>
+    </rules>
+  </firewall>
+</firewalls>
+ +
+
+ +

Get the details of a specified firewall

+ +

+To retrieve details of a single specified firewall use call GET /api/firewalls/:id. +

+ +

Example request:

+ +
+GET /api/firewalls/test?format=xml HTTP/1.1
+Authorization: Basic AU1J3UB2121Afd1DdyQWxLaTYTmJMNF4zTXBoRGdhMDh2RUw5ZDAN9zVXVa==
+User-Agent: curl/7.20.1 (i386-redhat-linux-gnu)
+Host: localhost:3001
+Accept: */*
+
+ +

Server reponse:

+ +
+HTTP/1.1 200 OK
+Content-Type: application/xml
+Date: Wed, 27 Jul 2011 08:20:29 GMT
+Content-Length: 835
+
+<?xml version='1.0' encoding='utf-8' ?>
+<firewall href='http://localhost:3001/api/firewalls/test' id='test'>
+  <name><![CDATA[test]]></name>
+  <description><![CDATA[this is just a test]]></description>
+  <owner_id>393485797142</owner_id>
+  <rules>
+    <rule href='http://localhost:3001/api/firewalls/test/393485797142~tcp~22~22~@group,393485797142,default,@address,ipv4,10.1.2.3,24' id='393485797142~tcp~22~22~@group,393485797142,default,@address,ipv4,10.1.2.3,24'>
+      <allow_protocol>tcp</allow_protocol>
+      <port_from>22</port_from>
+      <port_to>22</port_to>
+      <direction>ingress</direction>
+      <sources>
+        <source name='default' owner='393485797142' type='group'></source>
+        <source address='10.1.2.3' family='ipv4' prefix='24' type='address'></source>
+      </sources>
+    </rule>
+  </rules>
+</firewall>
+
+ +
+
+ +

Create a new firewall

+ +

+To create a new firewall use call GET /api/firewalls/:id. Clients must specify the firewall name and description as parameters to the request. The Deltacloud server will respond with HTTP 201 Created to a succesful completion and return details of the newly created firewall. As with other POST operations in the Deltacloud API, a client may specify parameters as multipart/form-data or using the application/x-www-form-urlencoded content-type. +

+ +

Example request:

+ +
+POST /api/firewalls?format=xml HTTP/1.1
+Authorization: Basic AU1J3UB2121Afd1DdyQWxLaTYTmJMNF4zTXBoRGdhMDh2RUw5ZDAN9zVXVa==
+User-Agent: curl/7.20.1 (i386-redhat-linux-gnu)
+Host: localhost:3001
+Accept: */*
+Content-Length: 64
+Content-Type: application/x-www-form-urlencoded
+
+name=Devel_Group&description=Access%20for%20all%20development%20machines
+
+ +

Server response:

+ +
+HTTP/1.1 201 Created
+Content-Type: application/xml
+Date: Wed, 27 Jul 2011 08:35:43 GMT
+Content-Length: 296
+
+<?xml version='1.0' encoding='utf-8' ?>
+<firewall href='http://localhost:3001/api/firewalls/Devel_Group' id='Devel_Group'>
+  <name><![CDATA[Devel_Group]]></name>
+  <description><![CDATA[Access for all development machines]]></description>
+  <owner_id></owner_id>
+  <rules>
+  </rules>
+</firewall>
+
+ +

Delete a firewall

+ +

+To delete the specified firewall from the back-end cloud provider use call DELETE /api/firewalls/:id. The Deltacloud server will respond with HTTP 204 No Content after a successful deletion: +

+ +

Example request:

+ +
+DELETE /api/firewalls/test?format=xml HTTP/1.1
+Authorization: Basic AU1J3UB2121Afd1DdyQWxLaTYTmJMNF4zTXBoRGdhMDh2RUw5ZDAN9zVXVa==
+User-Agent: curl/7.20.1 (i386-redhat-linux-gnu)
+Host: localhost:3001
+Accept: */*
+
+ +

Client response:

+ +
+HTTP/1.1 204 No Content
+Date: Wed, 27 Jul 2011 09:47:43 GMT
+
+ +

+The rules governing the deletion of a firewall are back-end cloud specific. At present, only the Amazon EC2 cloud supports the firewalls collection, so we describe the firewall deletion rules for EC2 here. +

+ +

+It is permitted to delete a firewall that has rules defined within it, with two exceptions. You cannot delete a firewall if it is referenced by another firewall; for instance firewall_1 has a rule giving access to firewall_2. An attempt to delete firewall_2 will result in the error InvalidGroup.InUse, as you can see in the example below. The second exception is that you cannot delete a firewall if there are currently any running instances within that firewall (i.e. instances that specified the given firewall when they were launched). The error message in that case would be InvalidGroup.InUse: There are active instances using security group. In both cases the error messages are propagated from the back-end cloud provider to the requesting client. +

+ +

+Example request (error deleting a firewall referenced by another firewall): +

+ +
+DELETE /api/firewalls/firewall_2?format=xml HTTP/1.1
+Authorization: Basic AU1J3UB2121Afd1DdyQWxLaTYTmJMNF4zTXBoRGdhMDh2RUw5ZDAN9zVXVa==
+User-Agent: curl/7.20.1 (i386-redhat-linux-gnu)
+Host: localhost:3001
+Accept: */*
+
+ +

Server response:

+ +
+HTTP/1.1 502 Bad Gateway
+Content-Type: application/xml
+Content-Length: 626
+
+<error status='502' url='/api/firewalls/firewall_2?format=xml'>
+  <kind>backend_error</kind>
+  <backend driver='ec2'>
+    <code>502</code>
+  </backend>
+  <message><![CDATA[InvalidGroup.InUse: Group 393485797142:firewall_2 is used by groups: 393485797142:firewall_1
+  REQUEST=ec2.us-east-1.amazonaws.com:443/?AWSAccessKeyId=AGG332FWWR5A11F327Q&Action=DeleteSecurityGroup&GroupName=firewall_2&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2011-07-27T08%3A50%3A50.000Z&Version=2010-08-31&Signature=g613223efwv5WAVhAosmPcrsfHqApAuw\nnwfYnZp0k3U%3D
+  REQUEST ID=8591fa20-a6ee-4db7-b30f-3022ecc9a9f5]]></message>
+</error>
+
+ +
+
+ +

Create a firewall rule

+ +

+To create a new firewall rule within a specified firewall use call POST /api/firewalls/:id/rules. A client must supply the protocol (one of udp, tcp or icmp), port_from and port_to as parameters. Of course the client must also specify the sources to which the given rule is applied. IP addresses are specified in CIDR format sequentially: ip_address1=192.168.10.10/24, ip_address2=10.1.1.1/16 ... ip_addressN=.... The IP address '0.0.0.0/0' acts as a wildcard to specify any IP address. Source firewalls are also specified sequentially, but the owner_id of the firewall prepared for authorization must also be supplied (this is an Amazon EC2 requirement): group1=name1, group1owner=1234567890, group2=name2, group2owner=0987654321, ... groupN=nameN, groupNowner=... +

+ +

+The Deltacloud server responds with a HTTP 201 Created after a successful operation and adds the details of the given firewall. The example client request below specifies the required parameters as multipart/form-data. However clients may also legitimately use the application/x-www-form-urlencoded to provide firewall rule parameters. +

+ +

Example request:

+ +
+POST /api/firewalls/default/rules?format=xml HTTP/1.1
+Authorization: Basic AU1J3UB2121Afd1DdyQWxLaTYTmJMNF4zTXBoRGdhMDh2RUw5ZDAN9zVXVa==
+User-Agent: curl/7.20.1 (i386-redhat-linux-gnu)
+Host: localhost:3001
+Accept: */*
+Content-Length: 1005
+Expect: 100-continue
+Content-Type: multipart/form-data; boundary=----------------------------4c9e7fa0a35e
+
+------------------------------4c9e7fa0a35e
+Content-Disposition: form-data; name="protocol"
+
+tcp
+------------------------------4c9e7fa0a35e
+Content-Disposition: form-data; name="port_from"
+
+22
+------------------------------4c9e7fa0a35e
+Content-Disposition: form-data; name="port_to"
+
+22
+------------------------------4c9e7fa0a35e
+Content-Disposition: form-data; name="group1"
+
+devel_group
+------------------------------4c9e7fa0a35e
+Content-Disposition: form-data; name="group1owner"
+
+393485797142
+------------------------------4c9e7fa0a35e
+Content-Disposition: form-data; name="group2"
+
+outside
+------------------------------4c9e7fa0a35e
+Content-Disposition: form-data; name="group2owner"
+
+393485797142
+------------------------------4c9e7fa0a35e
+Content-Disposition: form-data; name="ip_address1"
+
+192.168.1.1/24
+------------------------------4c9e7fa0a35e
+Content-Disposition: form-data; name="ip_address2"
+
+65.128.31.27/32
+------------------------------4c9e7fa0a35e--
+
+ +

Server response:

+ +
+HTTP/1.1 201 Created
+Content-Type: application/xml
+Date: Wed, 27 Jul 2011 10:18:51 GMT
+Content-Length: 1143
+
+<firewall href='http://localhost:3001/api/firewalls/default' id='default'>
+  <name><![CDATA[default]]></name>
+  <description><![CDATA[default group]]></description>
+  <owner_id>393485797142</owner_id>
+  <rules>
+    <rule href='http://localhost:3001/api/firewalls/default/393485797142~tcp~22~22~@group,393485797142,devel_group,@group,393485797142,outside,@address,ipv4,192.168.1.1,24,@address,ipv4,65.128.31.27,32' id='393485797142~tcp~22~22~@group,393485797142,devel_group,@group,393485797142,outside,@address,ipv4,192.168.1.1,24,@address,ipv4,65.128.31.27,32'>
+      <allow_protocol>tcp</allow_protocol>
+      <port_from>22</port_from>
+      <port_to>22</port_to>
+      <direction>ingress</direction>
+      <sources>
+        <source name='devel_group' owner='393485797142' type='group'></source>
+        <source name='outside' owner='393485797142' type='group'></source>
+        <source address='192.168.1.1' family='ipv4' prefix='24' type='address'></source>
+        <source address='65.128.31.27' family='ipv4' prefix='32' type='address'></source>
+      </sources>
+    </rule>
+  </rules>
+</firewall>
+
+ +

Delete a firewall rule

+ +

+To delete the specified firewall rule use call DELETE /api/firewalls/:id/:rule_id. The Deltacloud server will respond with HTTP 204 No Content on completion of a successful delete operation: +

+ +

Example request:

+ +
+DELETE /api/firewalls/default/393485797142~tcp~0~0~@group,393485797142,devel_group,@group,393485797142,outside,@address,ipv4,192.168.1.1,24,@address,ipv4,65.128.31.27,32?format=xml HTTP/1.1
+Authorization: Basic AU1J3UB2121Afd1DdyQWxLaTYTmJMNF4zTXBoRGdhMDh2RUw5ZDAN9zVXVa==
+User-Agent: curl/7.20.1 (i386-redhat-linux-gnu)
+Host: localhost:3001
+Accept: */*
+
+ +

Server response:

+ +
+HTTP/1.1 204 No Content
+Date: Wed, 27 Jul 2011 10:39:52 GMT
+
+ +
+
+ +Addresses + +