cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "SuichII, Christopher" <>
Subject Re: [PROPOSAL] Storage Subsystem API Interface Additions
Date Thu, 19 Sep 2013 12:16:06 GMT
John - any chance we can get your input on the original topic. Mikes comment was a kind of
unrelated (but a completely valid topic that I'd like to be a part of discussing anyway!).

Mike - are you planning on implementing the snapshotting methods of the storage subsystem
API anytime in the near future?

Kelvin or Darren, do you have thoughts on how thoughts on how this will work regarding the
user experience? As a user, I would be quite annoyed if I had to back up all of my vms one
at a a time. I guess I'm still not sure I understand the opposition to allowing users to perform
an action on more than one object at a time. I know that is kind of a foreign concept in CloudStack
right now, as the UI doesn't support it, but that is something I am currently working on any
(extending the UI to support this). If it is strictly a matter of making sure the user's expectations
regarding consistency are in line with what happens, then I think that is a separate discussion.

Maybe others would find it useful to have an API like this exposed if they are interacting
with CloudStack directly through the APIs rather than the UI. Imagine being a developer consuming
our APIs. It certainly seems cleaner (at least to me), to let the user call one API with a
list of vms to backup them all up instead of them requiring to loop around as many vms as
they want and having to call the same API that many times. 

To be perfectly clear, Kelvin, the benefit to propagating the ability to backup multiple vm
volumes at once to the UI is about ease of use for users and ease of development for us -
us being CS developers, CS plugin developers and CS API consumers.

Chris Suich
NetApp Software Engineer
Data Center Platforms – Cloud Solutions
Citrix, Cisco & Red Hat

On Sep 19, 2013, at 1:00 AM, Kelven Yang <> wrote:

> On 9/18/13 9:10 AM, "Darren Shepherd" <> wrote:
>> Here's my general concern about multiple volume snapshots at once.  Giving
>> such a feature leads the user to believe that snapshotting multiple
>> volumes
>> at once will give them consistency across the volumes in the snapshot.
>> This is not true, and difficult to do with many hypervisors, and typically
>> requires an agent in the VM.  A single snapshot, as exists today, is
>> really
>> crash consistent, meaning that there is may exist unsync'd data.  To do a
>> true multi volume snapshot requires a "quiesce" functionality in the VM.
>> So you do pause I/O queues, fsync, fsync, snapshot, snapshot, unpause I/O.
>> I'm might be fine with the option of allowing multiple volumeId's to be
>> specified in the snapshot API, but it needs to be clear that those
>> snapshots may be taken sequentially and they are all independently crash
>> consistent.  But, if you make that clear, then why even have the API.
>> Essentially it is the same as doing multiple snapshot API commands.
>> So really I would lean towards having the multiple snapshotting supported
>> in the driver or storage subsystem, but not exposed to the user.  You can
>> easy accomplish it by having a timed window on snapshotting.  So every 10
>> seconds you do snapshots, if 5 requests have queued in the last 10
>> seconds,
>> you do them all at once.  This could be implemented as a framework thing.
>> If your provider implements "SnapshotBatching" interface and that has a
>> getBatchWindowTime(), then the framework can detect that it should try to
>> queue up some snapshot requests and send them to the driver in a batch.
>> Or
>> that could be implemented in the driver itself.
> It makes more sense to me that "SnapshotBatching" is made available at
> storage framework layer for similar drivers that have such batch
> capability to share. There also exists another potential intelligent
> processing -  when storage subsystem layer processes independent
> volume-snapshot requests falling into the window, it can aggregate the
> requests targeting for the same VM instance into groups, this can allow
> hypervisor level drivers to take advantage of hypervisor provided VM
> snapshot wisely. 
> So +1 for storage layer - driver interface enhancements like this, but I
> don't see much immediate benefit to propagate it into end-user API layer.
> -Kelven
>> I would lean toward doing
>> it in the driver and if that goes well, we look at pulling the
>> functionality into core ACS.
>> Darren
>> On Wed, Sep 18, 2013 at 5:22 AM, SuichII, Christopher <
>>> wrote:
>>> I would like to raise for discussion the idea of adding a couple methods
>>> to the Storage Subsystem API interface. Currently, takeSnapshot() and
>>> revertSnapshot() only support single VM volumes. We have a use case for
>>> snapshotting multiple VM volumes at the same time. For us, it is more
>>> efficient to snapshot them all at once rather than snapshot VM Volumes
>>> individually and this seems like a more elegant solution than queueing
>>> the
>>> requests within our plugin.
>>> Base on my investigation, this should require:
>>> -Two additional API to be invoked from the UI
>>> -Two additional methods added to the Storage Subsystem API interface
>>> -Changes in between the API level and invoking the Storage Subsystem API
>>> implementations (I know this is broad and vague), mainly around the
>>> SnapshotManger/Impl
>>> There are a couple topics we would like discussion on:
>>> -Would this be beneficial/detrimental/neutral to other storage
>>> providers?
>>> -How should we handle the addition of new methods to the Storage
>>> Subsystem
>>> API interface? Default them to throw an UnsupportedOperationException?
>>> Default to calling the single VM volume version multiple times?
>>> -Does anyone see any issues with allowing multiple snapshots to be taken
>>> at the same time or letting storage providers have a list of all the
>>> requested volumes to backup?
>>> Please let me know if I've missed any major topics for discussion or if
>>> anything needs clarification.
>>> Thanks,
>>> Chris
>>> --
>>> Chris Suich
>>> NetApp Software Engineer
>>> Data Center Platforms ­ Cloud Solutions
>>> Citrix, Cisco & Red Hat

View raw message