cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Tutkowski <mike.tutkow...@solidfire.com>
Subject Re: [PROPOSAL] ROOT volume detach - feature for CS 4.5
Date Fri, 28 Mar 2014 23:22:11 GMT
See...I don't know the history behind this, but for VMware, when we shut a
VM down, the config files remain, which is not how this works on XenServer
or KVM.

That is the "root" (pun intended) of our managed-storage problem here.


On Fri, Mar 28, 2014 at 5:20 PM, Mike Tutkowski <
mike.tutkowski@solidfire.com> wrote:

> Sure :) In the storage_pool table, there is a column called "managed". 1 =
> managed
>
>
> On Fri, Mar 28, 2014 at 5:18 PM, Alena Prokharchyk <
> Alena.Prokharchyk@citrix.com> wrote:
>
>>  Ok, then can you please tell me the way how to determine if the
>> corresponding storage is managed, by looking at CS DB entry?
>>
>>   For phase #1 of the feature, I will just implement it for the regular
>> storage in KVM/Xen/VmWare; and implement managed storage support some time
>> later.
>>
>>  -Alena.
>>
>>   From: Mike Tutkowski <mike.tutkowski@solidfire.com>
>> Date: Friday, March 28, 2014 at 4:15 PM
>>
>> To: Alena Prokharchyk <alena.prokharchyk@citrix.com>
>> Cc: "dev@cloudstack.apache.org" <dev@cloudstack.apache.org>, "
>> shadowsor@gmail.com" <shadowsor@gmail.com>
>> Subject: Re: [PROPOSAL] ROOT volume detach - feature for CS 4.5
>>
>>   Yes
>>
>>  With non-managed storage, the admin determines when to manually create
>> and delete datastores.
>>
>>  I think this will only be a problem with managed storage on VMware.
>>
>>
>> On Fri, Mar 28, 2014 at 5:14 PM, Alena Prokharchyk <
>> Alena.Prokharchyk@citrix.com> wrote:
>>
>>>  So it only affects managed storage?
>>>
>>>   From: Mike Tutkowski <mike.tutkowski@solidfire.com>
>>> Date: Friday, March 28, 2014 at 4:10 PM
>>> To: Alena Prokharchyk <alena.prokharchyk@citrix.com>
>>> Cc: "dev@cloudstack.apache.org" <dev@cloudstack.apache.org>, "
>>> shadowsor@gmail.com" <shadowsor@gmail.com>
>>>
>>> Subject: Re: [PROPOSAL] ROOT volume detach - feature for CS 4.5
>>>
>>>   Let me illustrate this with an example:
>>>
>>>  * User creates a VM whose root disk is placed on managed storage
>>>
>>>  * Storage plug-in creates a volume on its SAN
>>>
>>>  * VMware server resource creates a datastore based on the newly
>>> created SAN volume (let me stress that this datastore was created by the
>>> VMware server resource - not manually by an admin as would be the case for
>>> non-managed storage)
>>>
>>>  * Inside the datastore are placed the VMDK file (root disk) along with
>>> VM config files like VMX, NVRAM, etc.
>>>
>>>  * User detaches the root volume (the VMDK file and VM config files
>>> remain in the datastore)
>>>
>>>  * User attaches another root volume to the VM (the VMDK file is stored
>>> in a datastore different from the datastore where the VM config files
>>> reside, which is fine for now)
>>>
>>>  * User deletes and expunges the original root disk (this leads to the
>>> datastore the VMDK file is on being removed...as a side effect, you will
>>> also lose your VM config files), SAN volume is deleted, CloudStack volume
>>> is marked as deleted in the database
>>>
>>>
>>> On Fri, Mar 28, 2014 at 5:05 PM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com> wrote:
>>>
>>>> So, do you guys see my concern with VMware, though?
>>>>
>>>>  VMware is different from XenServer and KVM in that its VM config
>>>> files are stored in the datastore along side the root volume (in CloudStack
>>>> 4.3, for example).
>>>>
>>>>  If you switch the VM to use a VMDK file in a different datastore, the
>>>> config files will remain in the original datastore (unless we transfer them
>>>> ourselves to the new datastore).
>>>>
>>>>  If they remain in the original datastore and that disk is deleted
>>>> later, the datastore that contains that disk will be removed (along with
>>>> the VM config files that are new being used in conjunction with a disk in
>>>> another datastore).
>>>>
>>>>
>>>> On Fri, Mar 28, 2014 at 4:58 PM, Alena Prokharchyk <
>>>> Alena.Prokharchyk@citrix.com> wrote:
>>>>
>>>>>
>>>>>
>>>>> On 3/28/14, 3:50 PM, "Marcus" <shadowsor@gmail.com> wrote:
>>>>>
>>>>> > I see this feature as mainly just shuffling around object properties
>>>>> >in the database. I don't expect any major issues to arise with any
>>>>> >storage if an inactive "root" disk is marked as a "data" disk in
the
>>>>> >DB, for example. In the end, when you start a VM you're always going
>>>>> >to have a root disk in the vm instance object, and volumes that are
>>>>> >attached/detached are going to be passed as data disks (If I
>>>>> >understand correctly). It doesn't really matter to the storage drivers
>>>>> >if the volume object was previously of type root or data.
>>>>>
>>>>>  Correct. That¹s what I reflected in the spec. But I¹m going to test
>>>>> it on
>>>>> all major supported hypervisors - KVM/Xen/VmWare - anyway, just to be
>>>>> 100%
>>>>> sure nothing breaks.
>>>>>
>>>>>
>>>>>
>>>>> >
>>>>> >On Fri, Mar 28, 2014 at 12:48 PM, Alena Prokharchyk
>>>>> ><Alena.Prokharchyk@citrix.com> wrote:
>>>>> >> I will look into it more, Mike. vmWare indeed can be different.
>>>>> >>
>>>>> >> -Alena.
>>>>> >>
>>>>> >> From: Mike Tutkowski
>>>>> >><mike.tutkowski@solidfire.com<mailto:mike.tutkowski@solidfire.com>>
>>>>> >> Date: Friday, March 28, 2014 at 11:39 AM
>>>>> >> To: Alena Prokharchyk
>>>>> >><alena.prokharchyk@citrix.com<mailto:alena.prokharchyk@citrix.com>>
>>>>> >> Cc: "dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>"
>>>>> >><dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>>
>>>>> >> Subject: Re: [PROPOSAL] ROOT volume detach - feature for CS
4.5
>>>>> >>
>>>>> >> VMware is also different because when you shut a VMware VM down
from
>>>>> >>CloudStack, the VM still exists in vCenter Server (whereas for
>>>>> XenServer
>>>>> >>and KVM, the VM is gone).
>>>>> >>
>>>>> >> Since the life of a datastore that was created for managed storage
>>>>> is
>>>>> >>tied to the life of the CloudStack volume it stores, when the
>>>>> CloudStack
>>>>> >>volume is deleted, the datastore goes away, as well.
>>>>> >>
>>>>> >> If the datastore in question was automatically created to store
a
>>>>> root
>>>>> >>disk (alongside VM config files) and you switch the VM to another
>>>>> root
>>>>> >>disk (which has to necessarily be in another datastore), you
won't
>>>>> see a
>>>>> >>problem until the original root volume is expunged by CloudStack.
At
>>>>> >>this point, its datastore will go away along with your VM config
>>>>> files.
>>>>> >>
>>>>> >>
>>>>> >> On Fri, Mar 28, 2014 at 12:31 PM, Mike Tutkowski
>>>>> >><mike.tutkowski@solidfire.com<mailto:mike.tutkowski@solidfire.com>>
>>>>> >>wrote:
>>>>> >> Well, the reason I brought it up was mainly due to VMware.
>>>>> >>
>>>>> >> Let's use that as an example:
>>>>> >>
>>>>> >> I initiate the process of spinning up a VM based on managed
storage.
>>>>> >> A volume is dynamically created on a SAN.
>>>>> >> VmwareStorageProcessor dynamically creates a datastore to consume
>>>>> the
>>>>> >>newly created SAN volume.
>>>>> >> All VMware VM files (ex. VMX, NVRAM) are placed in the datastore
>>>>> >>alongside the VMDK file that represents the root volume.
>>>>> >>
>>>>> >> Now, let's say we want to detach this root volume and give the
VM a
>>>>> new
>>>>> >>root volume.
>>>>> >>
>>>>> >> The new root volume will necessarily be on a different datastore
>>>>> than
>>>>> >>the datastore of the previous root volume (because a datastore
>>>>> created
>>>>> >>to consume managed storage will have at most one VMDK file*).
>>>>> >>
>>>>> >> Is it going to be a problem that the VM's files (ex. VMX, NVRAM)
>>>>> are on
>>>>> >>one datastore, but its root disk is on another?
>>>>> >>
>>>>> >> I don't think it's really a problem until you go to delete the
>>>>> original
>>>>> >>root volume from CloudStack. At that point, its datastore will
be
>>>>> >>removed (including, of course, your VM's VMX, NVRAM, etc. files).
>>>>> >>
>>>>> >> This is not really a problem on XenServer because XenServer
does not
>>>>> >>store VM config files in the SR, so I think we're OK there.
>>>>> >>
>>>>> >> We should also be OK for KVM.
>>>>> >>
>>>>> >> * Technically it can have many if those other VMDK files are
delta
>>>>> >>snapshots, but they still - together - represent a single disk.
>>>>> >>
>>>>> >>
>>>>> >> On Fri, Mar 28, 2014 at 10:36 AM, Alena Prokharchyk
>>>>> >><Alena.Prokharchyk@citrix.com<mailto:Alena.Prokharchyk@citrix.com>>
>>>>> >>wrote:
>>>>> >> Mike, thank you for the explanation on managed storage.. As
far as I
>>>>> >>understand from your email, the main difference is instead of
>>>>> creating
>>>>> >>an SR on the PS, CloudStack will recognize pre-existing volume
>>>>> created
>>>>> >>outside of the CS. Am I correct?
>>>>> >>
>>>>> >> If so, I don't think there would be any difference. When root
volume
>>>>> >>detach happens, no storage attributes - path, clusterId - are
being
>>>>> >>changed. And we would apply the same set of checks to the root
volume
>>>>> >>attach, as for a dataDisk attach.
>>>>> >>
>>>>> >> -Alena.
>>>>> >>
>>>>> >> From: Mike Tutkowski
>>>>> >><mike.tutkowski@solidfire.com<mailto:mike.tutkowski@solidfire.com>>
>>>>> >> Date: Thursday, March 27, 2014 at 9:40 PM
>>>>> >> To: Alena Prokharchyk
>>>>> >><alena.prokharchyk@citrix.com<mailto:alena.prokharchyk@citrix.com>>
>>>>> >> Cc: "dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>"
>>>>> >><dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>>
>>>>> >> Subject: Re: [PROPOSAL] ROOT volume detach - feature for CS
4.5
>>>>> >>
>>>>> >> Hi Alena,
>>>>> >>
>>>>> >> I was wondering if you've taken "managed" storage into consideration
>>>>> >>for this?
>>>>> >>
>>>>> >> If you're unfamiliar with it, managed storage is named as such
>>>>> because
>>>>> >>CloudStack manages it on behalf of the admin (ex. dynamically
>>>>> creating
>>>>> >>SRs as needed).
>>>>> >>
>>>>> >> For example, when I add primary storage to CloudStack that is
based
>>>>> on
>>>>> >>the SolidFire SAN, I use the SolidFire plug-in, which is an example
>>>>> of
>>>>> >>managed storage.
>>>>> >>
>>>>> >> In this case, the primary storage represents a SAN as opposed
to a
>>>>> >>preallocated volume.
>>>>> >>
>>>>> >> When the time comes to, say, attach a data disk to a VM for
the
>>>>> first
>>>>> >>time, the SolidFire plug-in goes off to its SAN and dynamically
>>>>> creates
>>>>> >>a new volume on it (with the appropriate size and IOPS requirements).
>>>>> >>
>>>>> >> CloudStack has logic that recognizes managed storage.
>>>>> >>
>>>>> >> For example, for XenServer, its logic has been augmented to
>>>>> >>automatically create an SR based on the iSCSI target that was
>>>>> created on
>>>>> >>the SAN and to create a VDI within it that is attached to the
VM in
>>>>> >>question.
>>>>> >>
>>>>> >> The big takeaway is that each CloudStack volume here will be
>>>>> associated
>>>>> >>with a unique volume on a SAN and consumed as an SR (XenServer)
or
>>>>> >>datastore (ESX) (KVM handles this differently). In this situation,
>>>>> there
>>>>> >>is a 1:1 mapping between a SAN volume and an SR. No other VDIs
are
>>>>> >>stored on the SR except for the one representing this one CloudStack
>>>>> >>volume.
>>>>> >>
>>>>> >> That being the case, I was wondering what you thought of this
with
>>>>> >>regards to your root-volume-detach feature?
>>>>> >>
>>>>> >> If we don't want to look into this for 4.5, it might be best
to
>>>>> simply
>>>>> >>fail to detach a root volume from a VM if the volume is based
on
>>>>> managed
>>>>> >>storage or to fail to attach a bootable volume to a VM if it
is
>>>>> based on
>>>>> >>managed storage.
>>>>> >>
>>>>> >> Talk to you later,
>>>>> >> Mike
>>>>> >>
>>>>> >>
>>>>> >> On Tue, Mar 25, 2014 at 1:24 PM, Alena Prokharchyk
>>>>> >><Alena.Prokharchyk@citrix.com<mailto:Alena.Prokharchyk@citrix.com>>
>>>>> >>wrote:
>>>>> >> Mike,
>>>>> >>
>>>>> >> Volume has a template_id referencing vm_template table. Vm_template
>>>>> has
>>>>> >> bootable flag, so we will derive information from there.
>>>>> >> And sure, this information will not change if the root disk
is
>>>>> detached.
>>>>> >>
>>>>> >> On 3/25/14, 12:18 PM, "Mike Tutkowski"
>>>>> >><mike.tutkowski@solidfire.com<mailto:mike.tutkowski@solidfire.com>>
>>>>> >> wrote:
>>>>> >>
>>>>> >>>Hi Alena,
>>>>> >>>
>>>>> >>>I was wondering how we plan to keep track of the new "bootable"
>>>>> >>>property?
>>>>> >>>When we create a VM, would we just mark its root disk as
bootable
>>>>> and
>>>>> >>>then
>>>>> >>>that property becomes immutable (for the upgrade case, all
root
>>>>> disks
>>>>> >>>would
>>>>> >>>be marked as bootable)?
>>>>> >>>
>>>>> >>>I'm thinking we'd want to keep track of bootable disks even
when
>>>>> there
>>>>> >>>are
>>>>> >>>detached and turned into data disks. Is that what you had
in mind?
>>>>> >>>
>>>>> >>>Thanks!
>>>>> >>>Mike
>>>>> >>>
>>>>> >>>
>>>>> >>>On Tue, Mar 25, 2014 at 12:20 PM, Alena Prokharchyk <
>>>>> >>>Alena.Prokharchyk@citrix.com<mailto:Alena.Prokharchyk@citrix.com>>
>>>>> >>>wrote:
>>>>> >>>
>>>>> >>>> Here is the link to the corresponding FS (placed in
"4.5 Design
>>>>> >>>>documents"
>>>>> >>>> section)
>>>>> >>>>
>>>>> >>>>
>>>>> >>>>
>>>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/ROOT+volume+deta
>>>>> >>>>ch
>>>>> >>>>
>>>>> >>>> -Alena.
>>>>> >>>>
>>>>> >>>> From: Alena Prokharchyk
>>>>> >>>><alena.prokharchyk@citrix.com<mailto:alena.prokharchyk@citrix.com
>>>>> ><mail
>>>>> >>>>to:
>>>>> >>>> alena.prokharchyk@citrix.com<mailto:alena.prokharchyk@citrix.com
>>>>> >>>
>>>>> >>>> Date: Monday, March 24, 2014 at 11:37 AM
>>>>> >>>> To:
>>>>> >>>>"dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org
>>>>> ><mailto:dev
>>>>> >>>>@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>>"
<
>>>>> >>>>
>>>>> >>>>dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org
>>>>> ><mailto:dev@
>>>>> >>>>cloudstack.apache.org<mailto:dev@cloudstack.apache.org>>>
>>>>> >>>> Subject: [PROPOSAL] ROOT volume detach - feature for
CS 4.5
>>>>> >>>>
>>>>> >>>> I would like to propose a new feature for CS 4.5 - "ROOT
volume
>>>>> >>>>detach"
>>>>> >>>>-
>>>>> >>>> that enables support for following use cases:
>>>>> >>>>
>>>>> >>>> 1) Replace current ROOT volume with the new one for
 existing vm.
>>>>> >>>> 2) Case when ROOT volume of vm1 gets corrupted, and
you want to
>>>>> attach
>>>>> >>>>it
>>>>> >>>> to vm2 to run the recovery utils on it. With current
CS
>>>>> implemntation,
>>>>> >>>>you
>>>>> >>>> have to perform several steps - create snapshot of vm1's
volume,
>>>>> >>>>create
>>>>> >>>> volume from snapshot, attach volume to the vm2. New
implementation
>>>>> >>>>will
>>>>> >>>> merge it all to one step.
>>>>> >>>>
>>>>> >>>>
>>>>> >>>> With the planned implementation, once the ROOT volume
is
>>>>> detached, it
>>>>> >>>>can
>>>>> >>>> be attached to any existing vm (with respect to
>>>>> Admin/Domain/Physical
>>>>> >>>> resources limitations), either as a DataDisk or a Root
disk.
>>>>> >>>>
>>>>> >>>> Amazon EC2 already has this functionality in place,
so I think CS
>>>>> >>>>would
>>>>> >>>> only benefit from having it. Storage experts (Edison,
others)
>>>>> please
>>>>> >>>>raise
>>>>> >>>> your concerns if you have any, or if you see any potential
>>>>> problems
>>>>> >>>>with
>>>>> >>>> the planned implementation. And if anyone can think
of other use
>>>>> cases
>>>>> >>>>this
>>>>> >>>> feature can possible solve, I would appreciate this
input as well.
>>>>> >>>>
>>>>> >>>>
>>>>> >>>> Feature limitations:
>>>>> >>>>
>>>>> >>>> * ROOT volume can be detached only when vm is in Stopped
state
>>>>> >>>> * CS will fail to start the vm not having a ROOT volume
>>>>> >>>>
>>>>> >>>> I will send out the link to the FS once I start getting
feedback
>>>>> on
>>>>> >>>>the
>>>>> >>>> proposal.
>>>>> >>>>
>>>>> >>>> -Alena.
>>>>> >>>>
>>>>> >>>
>>>>> >>>
>>>>> >>>
>>>>> >>>--
>>>>> >>>*Mike Tutkowski*
>>>>> >>>*Senior CloudStack Developer, SolidFire Inc.*
>>>>> >>>e: mike.tutkowski@solidfire.com<mailto:mike.tutkowski@solidfire.com
>>>>> >
>>>>> >>>o: 303.746.7302<tel:303.746.7302>
>>>>> >>>Advancing the way the world uses the
>>>>> >>>cloud<http://solidfire.com/solution/overview/?video=play>
>>>>> >>>*(tm)*
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >> --
>>>>> >> Mike Tutkowski
>>>>> >> Senior CloudStack Developer, SolidFire Inc.
>>>>> >> e: mike.tutkowski@solidfire.com<mailto:mike.tutkowski@solidfire.com
>>>>> >
>>>>> >> o: 303.746.7302<tel:303.746.7302>
>>>>> >> Advancing the way the world uses the
>>>>> >>cloud<http://solidfire.com/solution/overview/?video=play>(tm)
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >> --
>>>>> >> Mike Tutkowski
>>>>> >> Senior CloudStack Developer, SolidFire Inc.
>>>>> >> e: mike.tutkowski@solidfire.com<mailto:mike.tutkowski@solidfire.com
>>>>> >
>>>>> >> o: 303.746.7302<tel:303.746.7302>
>>>>> >> Advancing the way the world uses the
>>>>> >>cloud<http://solidfire.com/solution/overview/?video=play>(tm)
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >> --
>>>>> >> Mike Tutkowski
>>>>> >> Senior CloudStack Developer, SolidFire Inc.
>>>>> >> e: mike.tutkowski@solidfire.com<mailto:mike.tutkowski@solidfire.com
>>>>> >
>>>>> >> o: 303.746.7302
>>>>> >> Advancing the way the world uses the
>>>>> >>cloud<http://solidfire.com/solution/overview/?video=play>(tm)
>>>>>
>>>>>
>>>>
>>>>
>>>>  --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkowski@solidfire.com
>>>> o: 303.746.7302
>>>>  Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>>>> *(tm)*
>>>>
>>>
>>>
>>>
>>>  --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>>> *(tm)*
>>>
>>
>>
>>
>>  --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *(tm)*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *(tm)*
>



-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*(tm)*

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message