cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Tutkowski <mike.tutkow...@solidfire.com>
Subject Re: [PROPOSAL] Storage Subsystem API Interface Additions
Date Thu, 19 Sep 2013 14:24:12 GMT
I'm not sure on a release for when I'll implement snapshot functionality.
Maybe 4.4.


On Thu, Sep 19, 2013 at 6:16 AM, SuichII, Christopher <
Chris.Suich@netapp.com> wrote:

> John - any chance we can get your input on the original topic. Mikes
> comment was a kind of unrelated (but a completely valid topic that I'd like
> to be a part of discussing anyway!).
>
> Mike - are you planning on implementing the snapshotting methods of the
> storage subsystem API anytime in the near future?
>
> Kelvin or Darren, do you have thoughts on how thoughts on how this will
> work regarding the user experience? As a user, I would be quite annoyed if
> I had to back up all of my vms one at a a time. I guess I'm still not sure
> I understand the opposition to allowing users to perform an action on more
> than one object at a time. I know that is kind of a foreign concept in
> CloudStack right now, as the UI doesn't support it, but that is something I
> am currently working on any (extending the UI to support this). If it is
> strictly a matter of making sure the user's expectations regarding
> consistency are in line with what happens, then I think that is a separate
> discussion.
>
> Maybe others would find it useful to have an API like this exposed if they
> are interacting with CloudStack directly through the APIs rather than the
> UI. Imagine being a developer consuming our APIs. It certainly seems
> cleaner (at least to me), to let the user call one API with a list of vms
> to backup them all up instead of them requiring to loop around as many vms
> as they want and having to call the same API that many times.
>
> To be perfectly clear, Kelvin, the benefit to propagating the ability to
> backup multiple vm volumes at once to the UI is about ease of use for users
> and ease of development for us - us being CS developers, CS plugin
> developers and CS API consumers.
>
> --
> Chris Suich
> chris.suich@netapp.com
> NetApp Software Engineer
> Data Center Platforms – Cloud Solutions
> Citrix, Cisco & Red Hat
>
> On Sep 19, 2013, at 1:00 AM, Kelven Yang <kelven.yang@citrix.com> wrote:
>
> >
> >
> > On 9/18/13 9:10 AM, "Darren Shepherd" <darren.s.shepherd@gmail.com>
> wrote:
> >
> >> Here's my general concern about multiple volume snapshots at once.
>  Giving
> >> such a feature leads the user to believe that snapshotting multiple
> >> volumes
> >> at once will give them consistency across the volumes in the snapshot.
> >> This is not true, and difficult to do with many hypervisors, and
> typically
> >> requires an agent in the VM.  A single snapshot, as exists today, is
> >> really
> >> crash consistent, meaning that there is may exist unsync'd data.  To do
> a
> >> true multi volume snapshot requires a "quiesce" functionality in the VM.
> >> So you do pause I/O queues, fsync, fsync, snapshot, snapshot, unpause
> I/O.
> >>
> >> I'm might be fine with the option of allowing multiple volumeId's to be
> >> specified in the snapshot API, but it needs to be clear that those
> >> snapshots may be taken sequentially and they are all independently crash
> >> consistent.  But, if you make that clear, then why even have the API.
> >> Essentially it is the same as doing multiple snapshot API commands.
> >>
> >> So really I would lean towards having the multiple snapshotting
> supported
> >> in the driver or storage subsystem, but not exposed to the user.  You
> can
> >> easy accomplish it by having a timed window on snapshotting.  So every
> 10
> >> seconds you do snapshots, if 5 requests have queued in the last 10
> >> seconds,
> >> you do them all at once.  This could be implemented as a framework
> thing.
> >> If your provider implements "SnapshotBatching" interface and that has a
> >> getBatchWindowTime(), then the framework can detect that it should try
> to
> >> queue up some snapshot requests and send them to the driver in a batch.
> >> Or
> >> that could be implemented in the driver itself.
> >
> > It makes more sense to me that "SnapshotBatching" is made available at
> > storage framework layer for similar drivers that have such batch
> > capability to share. There also exists another potential intelligent
> > processing -  when storage subsystem layer processes independent
> > volume-snapshot requests falling into the window, it can aggregate the
> > requests targeting for the same VM instance into groups, this can allow
> > hypervisor level drivers to take advantage of hypervisor provided VM
> > snapshot wisely.
> >
> > So +1 for storage layer - driver interface enhancements like this, but I
> > don't see much immediate benefit to propagate it into end-user API layer.
> >
> > -Kelven
> >
> >
> >> I would lean toward doing
> >> it in the driver and if that goes well, we look at pulling the
> >> functionality into core ACS.
> >>
> >> Darren
> >>
> >>
> >> On Wed, Sep 18, 2013 at 5:22 AM, SuichII, Christopher <
> >> Chris.Suich@netapp.com> wrote:
> >>
> >>> I would like to raise for discussion the idea of adding a couple
> methods
> >>> to the Storage Subsystem API interface. Currently, takeSnapshot() and
> >>> revertSnapshot() only support single VM volumes. We have a use case for
> >>> snapshotting multiple VM volumes at the same time. For us, it is more
> >>> efficient to snapshot them all at once rather than snapshot VM Volumes
> >>> individually and this seems like a more elegant solution than queueing
> >>> the
> >>> requests within our plugin.
> >>>
> >>> Base on my investigation, this should require:
> >>> -Two additional API to be invoked from the UI
> >>> -Two additional methods added to the Storage Subsystem API interface
> >>> -Changes in between the API level and invoking the Storage Subsystem
> API
> >>> implementations (I know this is broad and vague), mainly around the
> >>> SnapshotManger/Impl
> >>>
> >>> There are a couple topics we would like discussion on:
> >>> -Would this be beneficial/detrimental/neutral to other storage
> >>> providers?
> >>> -How should we handle the addition of new methods to the Storage
> >>> Subsystem
> >>> API interface? Default them to throw an UnsupportedOperationException?
> >>> Default to calling the single VM volume version multiple times?
> >>> -Does anyone see any issues with allowing multiple snapshots to be
> taken
> >>> at the same time or letting storage providers have a list of all the
> >>> requested volumes to backup?
> >>>
> >>> Please let me know if I've missed any major topics for discussion or if
> >>> anything needs clarification.
> >>>
> >>> Thanks,
> >>> Chris
> >>> --
> >>> Chris Suich
> >>> chris.suich@netapp.com
> >>> NetApp Software Engineer
> >>> Data Center Platforms ­ Cloud Solutions
> >>> Citrix, Cisco & Red Hat
> >>>
> >>>
> >
>
>


-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message