incubator-cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "John Burwell" <jburw...@basho.com>
Subject Re: Review Request: S3-backed Secondary Storage
Date Fri, 21 Dec 2012 12:10:45 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8123/
-----------------------------------------------------------

(Updated Dec. 21, 2012, 12:10 p.m.)


Review request for cloudstack and edison su.


Changes
-------

Fixes a merge conflict on pom.xml.  The change in the previous patch attempt to fix an upstream
merge conflict.  On further review, cloudstack/master's content was identical.  Therefore,
the change was reverted to masteer's version.


Description
-------

Backs NFS-based secondary storage with an S3-compatible object store. Periodically, a reaper
thread synchronizes templates and ISOs stored on a NFS secondary storage mount with a configured
S3 object store. It also pushes snapshots to the object store when they are created and downloads
them in other zones on-demand. In addition to permitting the use of commodity or IaaS storage
solutions for static assets, it provides a means of automatically synchronizing template and
ISO assets across multiple zones.

For more information about the design of the patch, please see the design document (https://cwiki.apache.org/confluence/display/CLOUDSTACK/S3-backed+Secondary+Storage).


This addresses bug CLOUDSTACK-509.


Diffs (updated)
-----

  api/src/com/cloud/api/commands/ListS3sCmd.java PRE-CREATION 
  build/package.xml 09ed939 
  server/src/com/cloud/configuration/ConfigurationManagerImpl.java ef940e8 
  server/src/com/cloud/server/ManagementServerImpl.java 117be57 
  server/src/com/cloud/storage/dao/VMTemplateDao.java f5b6913 
  server/src/com/cloud/storage/dao/VMTemplateDaoImpl.java 2a0dfc8 
  server/src/com/cloud/storage/s3/S3ManagerImpl.java PRE-CREATION 
  tools/marvin/marvin/cloudstackConnection.py c805213 
  tools/marvin/marvin/configGenerator.py 11fbce9 
  tools/marvin/marvin/deployDataCenter.py bdf08cc 
  utils/src/com/cloud/utils/db/DbUtil.java feef7b3 

Diff: https://reviews.apache.org/r/8123/diff/


Testing
-------

I am submitting patch to begin the feedback process while we complete integration testing.
 I have verified that it does not interfere with normal CloudStack operations when S3-backed
Secondary Storage is disabled (the default setting) .  I have successfully tested operation
of single zone template and ISO scenarios on devcloud described in the design document.  I
am currently working through some issues in our multi-zone test environment to complete all
scenarios described.  The following are the known deficiencies of the current implementation
which I plan to correct in a subsequent patch:

   * Cross zone garbage collection: When a global asset is deleted from one zone's secondary
storage, it is not deleted from the secondary storage of other zones which have downloaded
it from the object store
   * S3 Configuration Update: The API only supports adding an object store configuration.
 Users should be able to edit the access key, secret key, connection timeout. max error retries,
and socket timeout.
   * Multi-threaded Uploads: Permit the upload of multiple assets to the object store simultaneously
to decrease the propagation latency across all zones.


Screenshots
-----------

S3 Configuration Form
  https://reviews.apache.org/r/8123/s/13/
S3 Enable Menu on the Zone Tab
  https://reviews.apache.org/r/8123/s/14/


Thanks,

John Burwell


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message