incubator-cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "John Burwell" <>
Subject Re: Review Request: S3-backed Secondary Storage
Date Fri, 14 Dec 2012 23:48:24 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated Dec. 14, 2012, 11:48 p.m.)

Review request for cloudstack and edison su.


Backs NFS-based secondary storage with an S3-compatible object store. Periodically, a reaper
thread synchronizes templates and ISOs stored on a NFS secondary storage mount with a configured
S3 object store. It also pushes snapshots to the object store when they are created and downloads
them in other zones on-demand. In addition to permitting the use of commodity or IaaS storage
solutions for static assets, it provides a means of automatically synchronizing template and
ISO assets across multiple zones.

For more information about the design of the patch, please see the design document (

This addresses bug CLOUDSTACK-509.


  api/src/com/cloud/api/commands/ 507053d 
  build/package.xml 09ed939 
  pom.xml e5aa024 
  server/src/com/cloud/configuration/ 2d7dfe2 
  server/src/com/cloud/server/ 117be57 
  server/src/com/cloud/storage/dao/ 1284ba1 
  server/src/com/cloud/storage/dao/ 5c71f1b 
  server/src/com/cloud/storage/s3/ 6b07254 
  tools/marvin/marvin/ 8c4e325 
  tools/marvin/marvin/ 01235fe 
  utils/src/com/cloud/utils/db/ feef7b3 



I am submitting patch to begin the feedback process while we complete integration testing.
 I have verified that it does not interfere with normal CloudStack operations when S3-backed
Secondary Storage is disabled (the default setting) .  I have successfully tested operation
of single zone template and ISO scenarios on devcloud described in the design document.  I
am currently working through some issues in our multi-zone test environment to complete all
scenarios described.  The following are the known deficiencies of the current implementation
which I plan to correct in a subsequent patch:

   * Cross zone garbage collection: When a global asset is deleted from one zone's secondary
storage, it is not deleted from the secondary storage of other zones which have downloaded
it from the object store
   * S3 Configuration Update: The API only supports adding an object store configuration.
 Users should be able to edit the access key, secret key, connection timeout. max error retries,
and socket timeout.
   * Multi-threaded Uploads: Permit the upload of multiple assets to the object store simultaneously
to decrease the propagation latency across all zones.


S3 Configuration Form
S3 Enable Menu on the Zone Tab


John Burwell

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message