cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "John Burwell" <>
Subject Re: Review Request: S3-backed Secondary Storage
Date Fri, 14 Dec 2012 23:54:26 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated Dec. 14, 2012, 11:54 p.m.)

Review request for cloudstack and edison su.


Add parent diff for the sake of completeness.


Backs NFS-based secondary storage with an S3-compatible object store. Periodically, a reaper
thread synchronizes templates and ISOs stored on a NFS secondary storage mount with a configured
S3 object store. It also pushes snapshots to the object store when they are created and downloads
them in other zones on-demand. In addition to permitting the use of commodity or IaaS storage
solutions for static assets, it provides a means of automatically synchronizing template and
ISO assets across multiple zones.

For more information about the design of the patch, please see the design document (

This addresses bug CLOUDSTACK-509.

Diffs (updated)

  api/src/com/cloud/api/commands/ PRE-CREATION 
  build/package.xml 09ed939 
  pom.xml 4a4276e 
  server/src/com/cloud/configuration/ ef940e8 
  server/src/com/cloud/server/ 117be57 
  server/src/com/cloud/storage/dao/ f5b6913 
  server/src/com/cloud/storage/dao/ 2a0dfc8 
  server/src/com/cloud/storage/s3/ PRE-CREATION 
  tools/marvin/marvin/ c805213 
  tools/marvin/marvin/ bdf08cc 
  utils/src/com/cloud/utils/db/ feef7b3 



I am submitting patch to begin the feedback process while we complete integration testing.
 I have verified that it does not interfere with normal CloudStack operations when S3-backed
Secondary Storage is disabled (the default setting) .  I have successfully tested operation
of single zone template and ISO scenarios on devcloud described in the design document.  I
am currently working through some issues in our multi-zone test environment to complete all
scenarios described.  The following are the known deficiencies of the current implementation
which I plan to correct in a subsequent patch:

   * Cross zone garbage collection: When a global asset is deleted from one zone's secondary
storage, it is not deleted from the secondary storage of other zones which have downloaded
it from the object store
   * S3 Configuration Update: The API only supports adding an object store configuration.
 Users should be able to edit the access key, secret key, connection timeout. max error retries,
and socket timeout.
   * Multi-threaded Uploads: Permit the upload of multiple assets to the object store simultaneously
to decrease the propagation latency across all zones.


S3 Configuration Form
S3 Enable Menu on the Zone Tab


John Burwell

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message