incubator-cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Kinsella <>
Subject Re: new storage framework update
Date Fri, 28 Dec 2012 21:58:43 GMT
The reason I brought it up is I suspected there was a slight difference - should probably clarify
that in the wiki page, or use consistent wording.

On Dec 28, 2012, at 1:36 PM, Alex Huang <> wrote:

> I think John asked a great question and we should define them as different.
> When we talk about data motion, we are talking about a provisioning step where we hand
off a source uri and a destination uri and ask a service to move data from one location to
another.  The data can be template, iso, snapshot, volumes but to the service it's nothing
more than two uri that it has to understand.
> In terms of certain hypervisors, this data motion provisioning step may be performed
by their volume migration feature.  
> --Alex
>> -----Original Message-----
>> From: Edison Su []
>> Sent: Friday, December 28, 2012 12:17 PM
>> To:
>> Subject: RE: new storage framework update
>> Yes, it just means move one entity(volume/snapshot/template etc) from
>> one place to another.
>>> -----Original Message-----
>>> From: John Kinsella []
>>> Sent: Friday, December 28, 2012 11:40 AM
>>> To:
>>> Subject: Re: new storage framework update
>>> Great writeup! On the wiki page, are "motion" and "migration"
>>> interchangeable?
>>> John
>>> On Dec 27, 2012, at 6:41 PM, Edison Su <> wrote:
>>>> Hi All,
>>>>    Before heading into holiday, I'd like to update the current status of
>>> new storage framework since last collab12.
>>>>   1. Class diagram of primary storage is evolved:
>>> age.jpg?version=1&modificationDate=1356640617613
>>>>         Highlight the current design:
>>>>         a.  One storage provider can cover multiple storage protocols for
>>> multiple hypervisors. The default storage provider can almost cover all the
>>> current primary storage protocols. In most of cases, you don't need to
>> write a
>>> new storage provider, what you need to do is to write a new storage
>>> configurator. Write a new storage provider needs to write a lot of code,
>>> which we should avoid it as much as possible.
>>>>        b. A new type hierarchy, primaryDataStoreConfigurator, is added.
>> The
>>> configurator is a factory for primaryDataStore, which assemble
>>> StorageProtocolTransformer, PrimaryDataStoreLifeCycle and
>>> PrimaryDataStoreDriver for PrimaryDataStore object, based on the
>>> hypervisor type and the storage protocol.  For example, for nfs primary
>>> storage on xenserver, there is a class called XenNfsConfigurator, which put
>>> DefaultXenPrimaryDataStoreLifeCycle, NfsProtocolTransformer and
>>> DefaultPrimaryDataStoreDriverImpl into DefaultPrimaryDataStore. One
>>> provider can only have one configurator for a pair of hypervisor type and
>>> storage protocol. For example, if you want to add a new nfs protocol
>>> configurator for xenserver hypervisor, you need to write a new storage
>>> provider.
>>>>       c. A new interface, StorageProtocolTransformer, is added. The main
>>> purpose of this interface is to handle the difference between different
>>> storage protocols. It has four methods:
>>>>            getInputParamNames: return a list of name of parameters for a
>>> particular protocol. E.g. NFS protocol has ["server", "path"], ISCSI has ["iqn",
>>> "lun"] etc. UI shouldn't hardcode these parameters any more.
>>>>            normalizeUserInput: given a user input from UI/API, need to
>> validate
>>> the input, and break it apart, then store them into database
>>>>            getDataStoreTO/ getVolumeTO: each protocol can have its own
>>> volumeTO and primaryStorageTO. TO is the object will be passed down to
>>> resource, if your storage has extra information you want to pass to
>> resource,
>>> these two methods are the place you can override.
>>>>       d. All the time-consuming API calls related to storage is async.
>>>>      2. Minimal functionalities are implemented:
>>>>           a. Can register a http template, without SSVM
>>>>           b. Can register a NFS primary storage for xenserver
>>>>           c. Can download a template into primary storage directly
>>>>          d. Can create a volume from a template
>>>>      3. All about test:
>>>>          a. TestNG test framework is used, as it can provide parameter for
>> each
>>> test case. For integration test, we need to know ip address of hypervisor
>>> host, the host uuid(if it's xenserver), the primary storage url, the template
>> url
>>> etc. These configurations are better to be parameterized, so for each test
>>> run, we don't need to modify test case itself, instead, we provide a test
>>> configuration file for each test run. TestNG framework already has this
>>> functionality, I just reuse it.
>>>>          b. Every pieces of code can be unit tested, which means:
>>>>                b.1 the xcp plugin can be unit tested. I wrote a small python
>>> called, which can directly call xcp plugin.
>>>>                b.2 direct agent hypervisor resource can be tested. I wrote
>> mock
>>> agent manger, which can load and initialize hypervisor resource, and also
>> can
>>> send command to resource.
>>>>                b.3 a storage integration test maven project is created, which
>>> test the whole storage subsystem, such as create volume from template,
>>> which including both image and volume components.
>>>>          A new section, called "how to test", is added into
>>> em+2.0, please check it out.
>>>>     The code is on the javelin branch, the maven projects whose name
>>> starting from cloud-engine-storage-* are the code related to storage
>>> subsystem. Most of the primary storage code is in cloud-engine-storage-
>>> volume project.
>>>>      Any feedback/comment is appreciated.
>>> Stratosec - Secure Infrastructure as a Service
>>> o: 415.315.9385
>>> @johnlkinsella

Stratosec - Secure Infrastructure as a Service
o: 415.315.9385

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message