cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Edison Su <>
Subject [MERGE]object_store branch into master [Second round]
Date Sat, 15 Jun 2013 03:07:39 GMT
Hi all, 
     The second round, call for merge object_store branch into master, is coming!
      The issues fixed:
           1. All the major issues addressed by John are fixed:
                1.1 A cache storage replacement algorithm is added: StorageCacheReplacementAlgorithmLRU,
based on the reference count and least recently used. 
                1.2 A new S3 transport is added, can upload > 5G template into S3 directly.
                1.3 Retry, if S3 upload failed.
                1.4 some comments from, mostly, the coding
style, are addressed and clean up some unused code.
         2. DB upgrade path from 4.1 to 4.2
         3. Bug fix

    The size of the patch is even bigger now, around 100000 LOC, you can find about the patch
     Comments/feedback are welcome. Thanks.

> -----Original Message-----
> From: Edison Su []
> Sent: Friday, May 17, 2013 1:11 AM
> To:
> Subject: [MERGE]object_store branch into master
> Hi all,
>      Min and I worked on object_store branch during the last one and half
> month. We made a lot of refactor on the storage code, mostly related to
> secondary storage, but also on the general storage framework. The following
> goals are made:
> 1.       An unified storage framework. Both secondary storages(nfs/s3/swift
> etc) and primary storages will share the same plugin model, the same
> interface. Add any other new storages into cloudstack will much easier and
> straightforward.
> 2.       The storage interface  between mgt server and resource is unified,
> currently there are only 5 commands send out by mgt server:
> copycommand/createobjectcommand/deletecommand/attachcommand/de
> ttachcommand, and each storage vendor can decode/encode all the
> entities(volume/snapshot/storage pool/ template etc) by its own.
> 3.       NFS secondary storage is not explicitly depended on by other
> components. For example, when registering template into S3, template will
> be write into S3 directly, instead of storing into nfs secondary storage, then
> push to S3. If s3 is used as secondary storage, then nfs storage will be used as
> cache storage, but from other components point of view, cache storage is
> invisible. So, it's possible to make nfs storage as optional if s3 is used for
> certain hypervisors.
> The detailed FS is at
> +Object+Store+Plugin+Framework
> The test we did:
> 1.       We modified marvin to use new storage api
> 2.       Test_volume and test_vm_life_cycle, test_template under smoke test
> folder are executed against xenserver/kvm/vmware and devcloud, some of
> them are failed, it's partly due to bugs introduced by our code, partly master
> branch itself has issue(e.g. resizevolume doesn't work). We want to fix these
> issues after merging into master.
> The basic follow does work: create user vm, attach/detach volume, register
> template, create template from volume/snapshot, take snapshot, create
> volume from snapshot.
>   It's a huge change, around 60k LOC patch, to review the code, you can try:
> git diff master..object_store, will show all the diff.
>   Comments/feedback are welcome. Thanks.

View raw message