cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Tutkowski <mike.tutkow...@solidfire.com>
Subject Re: [Propose][New Feature] Storage Snapshots
Date Fri, 05 Feb 2016 02:02:28 GMT
I think that all sounds reasonable then - thanks!

On Thu, Feb 4, 2016 at 6:52 PM, Syed Mushtaq <syed1.mushtaq@gmail.com>
wrote:

> You are correct Mike in terms of the requirements. One of our earlier
> iterations on this was to have an argument to the create snapshot API which
> decides whether to backup the volume to sec storage but we realized it
> would make management of snapshots quite messy so we proposed a new api
> instead.
>
> On Thu, Feb 4, 2016, 8:29 PM Mike Tutkowski <mike.tutkowski@solidfire.com>
> wrote:
>
>> Hi,
>>
>> Just to make sure I understand all the requirements here:
>>
>> 1) This relates only to managed storage (1:1 mapping between a virtual
>> disk and a backend SAN volume).
>>
>> 2) We want to take the current (introduced in 4.6) functionality, which
>> creates a snapshot on the SAN, and extend it via a config option (or
>> something) to not only take the SAN snapshot, but to copy the underlying
>> VHD (XenServer only) to NFS.
>>
>> 3) The SAN snapshot is always taken. It's the backup to NFS that is
>> optional.
>>
>> 4) Templates can be created from the snapshot that's on the SAN (already
>> works).
>>
>> 5) CloudStack volumes can be created from the snapshot that's on the SAN
>> (already works as long as the new CloudStack volume ends up on the same
>> primary storage).
>>
>> Would we have a need for a storage snapshot API then or would that just
>> be the standard volume snapshot without the backup to NFS?
>>
>> Thanks!
>> Mike
>>
>> On Thu, Feb 4, 2016 at 5:24 PM, Syed Mushtaq <syed1.mushtaq@gmail.com>
>> wrote:
>>
>>> Is it possible to have both functionalities (snapshot on SAN & Sec
>>> Storage) coexist? Because Ideally, we would like to have both.
>>> For example, some of our customers want to implement their own backup
>>> strategies and do encryption to their backups which is a perfect
>>> use case for Storage Snapshot while our other customers will still keep
>>> using the standard volume snapshot.
>>>
>>> To keep things backward compatible, we can add a setting which says to
>>> not upload on secondary storage, because, after all, you would take a
>>> SAN snapshot first when doing a Volume Snapshot. You could stop the
>>> process there and not do the upload.
>>>
>>> What do you think about this approach?
>>>
>>> Thanks,
>>> -Syed
>>>
>>> On Thu, Feb 4, 2016 at 4:25 PM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com> wrote:
>>>
>>>> So, this is just me thinking out load here, but if a given CloudStack
>>>> cloud doesn't actually need to provide both the ability to take a SAN
>>>> snapshot and export it to NFS (if just taking a SAN snapshot is OK), then
>>>> we might be able to get away with no new API calls and simply implement a
>>>> new custom snapshot strategy and data motion strategy to handle the case
>>>> where the CloudStack cloud does want both a SAN snapshot and
>>>> exported-to-NFS backup.
>>>>
>>>> In other words, the "default" behavior would be to use the snapshot
>>>> strategy and data motion strategy that we already have (the one that only
>>>> takes a SAN snapshot).
>>>>
>>>> If your CloudStack cloud, however, wants to take a SAN snapshot and
>>>> have the data exported to NFS, then we could have you manipulate a Swing
>>>> config file to make use of a new snapshot strategy and data motion strategy
>>>> that performs both of these activities.
>>>>
>>>> This way, the old behavior is still the default for users, but
>>>> CloudStack admins can change this behavior via configuration.
>>>>
>>>> Thoughts?
>>>>
>>>> On Thu, Feb 4, 2016 at 11:55 AM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com> wrote:
>>>>
>>>>> Right...I think we will need to come up with a viable upgrade path or
>>>>> some reasonable way for them to move from the old way to the new way
(and
>>>>> some obvious way that they will know they need to do this).
>>>>>
>>>>> On Thu, Feb 4, 2016 at 11:45 AM, Syed Mushtaq <syed1.mushtaq@gmail.com
>>>>> > wrote:
>>>>>
>>>>>> I'm not really sure about the upgrade path however, customers who
are
>>>>>> using 4.6 and are on a managed storage would no longer have the same
>>>>>> functionality with Volume Snapshots.
>>>>>>
>>>>>> On Thu, Feb 4, 2016 at 1:43 PM, Syed Mushtaq <syed1.mushtaq@gmail.com
>>>>>> > wrote:
>>>>>>
>>>>>>> So if I understand correctly, currently taking a Volume Snapshots
of
>>>>>>> a volume on a managed storage keeps it on the storage array.
As a part of
>>>>>>> this feature, we can make sure that Volume Snapshots on managed
storage are
>>>>>>> uploaded to the secondary storage. This would make the Volume
Snapshot
>>>>>>> feature behave the same regardless of the storage (managed or
non-managed)
>>>>>>> And, for utilizing the efficient backend storage capabilities,
we can use
>>>>>>> the new storage snapshots API.
>>>>>>>
>>>>>>> On Thu, Feb 4, 2016 at 1:36 PM, Mike Tutkowski <
>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>
>>>>>>>> Hi everyone,
>>>>>>>>
>>>>>>>> Whatever we do here, we need to have a plan to deal with
the fact
>>>>>>>> that we already have a feature (in 4.6 and later) that allows
you to use
>>>>>>>> the existing volume-snapshot APIs to create a volume snapshot
(for managed
>>>>>>>> storage) that resides on a backend SAN (using a custom snapshot
strategy
>>>>>>>> and a custom data motion strategy).
>>>>>>>>
>>>>>>>> If these new APIs go in, then how should the original
>>>>>>>> implementation (present in 4.6 and later) be changed? If
it is changed, how
>>>>>>>> do we support customers who are already using the original
volume-snapshot
>>>>>>>> API to take snapshots on a backend SAN?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Mike
>>>>>>>>
>>>>>>>> On Thu, Feb 4, 2016 at 11:27 AM, Will Stevens <
>>>>>>>> wstevens@cloudops.com> wrote:
>>>>>>>>
>>>>>>>>> Will you be able to create a Template from a StorageSnapshot?
 If
>>>>>>>>> yes, will the template be stored in the secondary storage
like normal
>>>>>>>>> templates or will that be handled somehow on the vendor
side?
>>>>>>>>>
>>>>>>>>> *Will STEVENS*
>>>>>>>>> Lead Developer
>>>>>>>>>
>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
>>>>>>>>> w cloudops.com *|* tw @CloudOps_
>>>>>>>>>
>>>>>>>>> On Thu, Feb 4, 2016 at 1:22 PM, Syed Mushtaq <
>>>>>>>>> syed1.mushtaq@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Thanks Will!!!
>>>>>>>>>>
>>>>>>>>>> On Thu, Feb 4, 2016 at 1:19 PM, Will Stevens <
>>>>>>>>>> wstevens@cloudops.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> I explicitly linked the Design Spec in the Jira
ticket because
>>>>>>>>>>> it was not clear in the 'mention' section because
it shows as a page 'you
>>>>>>>>>>> do not have permission to'.
>>>>>>>>>>>
>>>>>>>>>>> *Will STEVENS*
>>>>>>>>>>> Lead Developer
>>>>>>>>>>>
>>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
>>>>>>>>>>> w cloudops.com *|* tw @CloudOps_
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Feb 4, 2016 at 1:02 PM, Syed Ahmed <sahmed@cloudops.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Design Spec:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/StorageSnapshot++API
>>>>>>>>>>>>
>>>>>>>>>>>> Jira Ticket
>>>>>>>>>>>>
>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-9278
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>
>>>>>>>>>>>> We plan to propose a new set of APIs to do
snapshots on managed
>>>>>>>>>>>> storage backends like SolidFire. Snapshots
on current managed storage stay
>>>>>>>>>>>> on the
>>>>>>>>>>>> device which is contrary to what CloudStack
calls snpshots. But
>>>>>>>>>>>> taking snapshots
>>>>>>>>>>>> on storage and keeping it there has its own
advantages and we
>>>>>>>>>>>> would ideally like
>>>>>>>>>>>> to have both ways of doing snapshots. This
proposal adds 4 new
>>>>>>>>>>>> APIs to
>>>>>>>>>>>> create snapshots on backend storage.
>>>>>>>>>>>>
>>>>>>>>>>>> What do you guys think of this feature? I
would love to have
>>>>>>>>>>>> some feedback. I am working on making the
design spec more concrete but
>>>>>>>>>>>> wanted to have
>>>>>>>>>>>> a high level feedback first before starting
to work on it.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> -Syed
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Mike Tutkowski*
>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>> o: 303.746.7302
>>>>>>>> Advancing the way the world uses the cloud
>>>>>>>> <http://solidfire.com/solution/overview/?video=play>*™*
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Mike Tutkowski*
>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> e: mike.tutkowski@solidfire.com
>>>>> o: 303.746.7302
>>>>> Advancing the way the world uses the cloud
>>>>> <http://solidfire.com/solution/overview/?video=play>*™*
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkowski@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the cloud
>>>> <http://solidfire.com/solution/overview/?video=play>*™*
>>>>
>>>
>>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud
>> <http://solidfire.com/solution/overview/?video=play>*™*
>>
>


-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the cloud
<http://solidfire.com/solution/overview/?video=play>*™*

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message