cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Tutkowski <mike.tutkow...@solidfire.com>
Subject Re: Storage Subsystem 2.0 plugin docs
Date Thu, 28 Mar 2013 21:39:50 GMT
Let me draw a diagram and send it to you guys.

The gist is that all storage plug-ins that make use of iSCSI will need
certain support (like communicating with the hypervisor of the cluster the
storage is for and hooking up said storage).  The storage plug-ins
themselves should not know anything about the hypervisor of the cluster
they're creating storage for.


On Thu, Mar 28, 2013 at 3:25 PM, Mike Tutkowski <
mike.tutkowski@solidfire.com> wrote:

> Hi Edison,
>
> Can you clarify what you mean here?
>
> [Edison] If there are HV-dependent storage code there(I assume it should
> have some code can be shared between different storage provider at
> hypervisor side), we can generalize them and expose them.
>
> I think Vladimir and I are proposing that the storage framework be
> modified to only expect its plug-ins to write code that deals with their
> array (not hypervisor-related code).  For example, the storage framework
> could call into my plug-in for a new volume, I would create it using the
> SolidFire API, return an IQN, and the storage framework would run the logic
> it needs to in order to, say, create a Datastore for VMware hosts based on
> that IQN.
>
> What do you think about that Edison?  Either way, I'm working on writing
> code that creates a VMware Datastore and we can decide where to place it
> (either in a utility shared by the storage plug-ins or in the CS storage
> framework).
>
>
> On Thu, Mar 28, 2013 at 2:12 PM, Edison Su <Edison.su@citrix.com> wrote:
>
>> Comments embedded in below.****
>>
>> ** **
>>
>> *From:* Vladimir Popovski [mailto:vladimir@zadarastorage.com]
>> *Sent:* Wednesday, March 27, 2013 12:22 PM
>> *To:* Mike Tutkowski; Edison Su
>> *Cc:* cloudstack-dev@incubator.apache.org
>>
>> *Subject:* RE: Storage Subsystem 2.0 plugin docs****
>>
>> ** **
>>
>> Hi All,****
>>
>>  ****
>>
>> I was away for couple of days, so sorry for the delay.****
>>
>>  ****
>>
>> I’m completely with Mike & John (& others) on separating storage plugins
>> from hypervisor-related functions. If every plugin will need to implement
>> hypervisor-related code, it will be a big mess.****
>>
>>  ****
>>
>> Our use-case is very similar to Mike’s – we would like to be able to
>> provision volumes with different QoS characteristics directly to VMs,
>> rather than adding them into “shared” datastores. It might be achieved in
>> two ways:****
>>
>> -          either to create separate data stores per each volume
>> (storage LUN), and from there to create volumes & assign to instances.***
>> *
>>
>> -          or to assign devices recognized by iSCSI Initiators directly
>> to instances (I’m not sure if it will be possible in VMware without
>> datastores)****
>>
>>  ****
>>
>> It looks like Mike started to work on the 1st approach. In this case,
>> for every volume there will be a separate datastore. If this is the
>> preferred direction for all storage plugins, then the hypervisor-specific
>> logic of datastore creation and creating/assigning volumes from the
>> datastore will be fairly common for all storage plug-ins. At the same time,
>> the storage plugin should have the control over the datastore management.
>> It will depend on the plugin, if dedicated or shared datastores should be
>> created.****
>>
>>  ****
>>
>> For the 2nd option (skipping the datastore layer) there might be plenty
>> of common code as well. ****
>>
>>  ****
>>
>> So, my questions are:****
>>
>> -          do you think it is beneficial to support both options for the
>> CS (or are we good with potentially huge number of datastores)?****
>>
>> ** **
>>
>> [Edison]  CloudStack will not enforce any of these options, it’s all up
>> to provider’s implementation. Either way is OK to me. Do you think, from
>> architecture point of view, Is the current storage API enough for both
>> options? If no, we can come up some new APIs.****
>>
>> ** **
>>
>> -          are we all agree that HV-dependent storage code should be
>> generic and appropriate interfaces to be exposed?****
>>
>> ** **
>>
>> [Edison] If there are HV-dependent storage code there(I assume it should
>> have some code can be shared between different storage provider at
>> hypervisor side), we can generalize them and expose them.****
>>
>> As Mike said, the code dealing with storage pool at the hypervisor side,
>> can be shared.****
>>
>>  ****
>>
>> Regards,****
>>
>> -Vladimir****
>>
>>  ****
>>
>>  ****
>>
>>  ****
>>
>> *From:* Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>> *Sent:* Wednesday, March 27, 2013 10:21 AM
>> *To:* Edison Su
>> *Cc:* cloudstack-dev@incubator.apache.org; Vladimir Popovski
>> *Subject:* Re: Storage Subsystem 2.0 plugin docs****
>>
>>  ****
>>
>> Sounds good, Edison****
>>
>>  ****
>>
>> Last night I finished up code that uses the VI Java API to create a
>> VMware Datastore.****
>>
>>  ****
>>
>> I want to test it a bit more before I have you look at it.****
>>
>>  ****
>>
>> Today there is a Citrix CloudPlatform demo I'm participating in to handle
>> part of the SolidFire section of the demo, so I might not have time to do
>> my Datastore testing, but I should be done with it tomorrow.****
>>
>>  ****
>>
>> Talk to you later!****
>>
>>  ****
>>
>> On Wed, Mar 27, 2013 at 11:15 AM, Edison Su <Edison.su@citrix.com> wrote:
>> ****
>>
>> For vmware, current cloudstack doesn’t create a vmware datastore through
>> vmware’s API, admin needs to create the datastore in Vcenter at first, then
>> add it back into cloudstack. I am not familiar with how to create a VMware
>> datastore through Vmware’s API, but regarding to add a new host into a
>> cluster, the current framework lets storage provider adding a listener
>> which can listen on adding host event. ****
>>
>>  ****
>>
>> *From:* Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>> *Sent:* Tuesday, March 26, 2013 6:45 PM****
>>
>>
>> *To:* Edison Su
>> *Cc:* cloudstack-dev@incubator.apache.org; Vladimir Popovski
>> *Subject:* Re: Storage Subsystem 2.0 plugin docs****
>>
>>  ****
>>
>> Great - thanks, Edison!****
>>
>>  ****
>>
>> I can take a look at that code.****
>>
>>  ****
>>
>> I've almost gotten the VMware code written.****
>>
>>  ****
>>
>> It's a little more involved than the XenServer code because you have to
>> add static IQNs for discovery to each host in a VMware cluster (this is
>> somehow handled behind the scenes, I suppose, with XenServer) before you
>> can create a Datastore based on your iSCSI target.****
>>
>>  ****
>>
>> One thing I was wondering, though, is when you add a new host to this
>> VMware cluster.  It will need to "inherit" the list of IQNs to discover.  I
>> image this is the case today.  Do you know anything about that?  I might
>> just try it out and see if that works today.****
>>
>>  ****
>>
>> On Tue, Mar 26, 2013 at 5:18 PM, Edison Su <Edison.su@citrix.com> wrote:*
>> ***
>>
>> Thanks! ****
>>
>> FYI, there are some code at both xen and kvm hypervisor resource code to
>> deal with storage pool creation. ****
>>
>> For example, in CitrixResourceBase-> getNfsSR or getIscsiSR to create a
>> nfs SR or ISCSI SR. In LibvirtStorageAdaptor, which can create storage pool
>> in libvirt.****
>>
>>  ****
>>
>>  ****
>>
>> *From:* Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>> *Sent:* Tuesday, March 26, 2013 1:52 PM
>> *To:* Edison Su
>> *Cc:* cloudstack-dev@incubator.apache.org; Vladimir Popovski****
>>
>>
>> *Subject:* Re: Storage Subsystem 2.0 plugin docs****
>>
>>  ****
>>
>> Hi Edison,****
>>
>>  ****
>>
>> Sounds good.****
>>
>>  ****
>>
>> I already have code to create a XenServer Storage Repository (and
>> optionally use CHAP credentials) and I'm working right now on creating a
>> vSphere Datastore.****
>>
>>  ****
>>
>> When I have this working and in a nicer state, I can check in with you to
>> review it and provide comments.****
>>
>>  ****
>>
>> Once those two hypervisors are handled, I'll move on to KVM and OVM.****
>>
>>  ****
>>
>> Thanks!****
>>
>>  ****
>>
>> On Tue, Mar 26, 2013 at 2:33 PM, Edison Su <Edison.su@citrix.com> wrote:*
>> ***
>>
>> Yes, code is welcome!!! Currently Only the interface at the management
>> server side is defined. At the hypervisor resource side, we may need some
>> kind of utility library or another plugin framework, as John proposed few
>> months ago.****
>>
>>  ****
>>
>> *From:* Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>> *Sent:* Monday, March 25, 2013 2:37 PM
>> *To:* Edison Su; cloudstack-dev@incubator.apache.org; Vladimir Popovski**
>> **
>>
>>
>> *Subject:* Re: Storage Subsystem 2.0 plugin docs****
>>
>>  ****
>>
>> Hey Edison,****
>>
>>  ****
>>
>> So...if you think my understanding is correct (please check out the
>> e-mail below), then I have a question.****
>>
>>  ****
>>
>> Do we really want to have the storage plug-ins taking on the
>> responsibility of talking to the hypervisors to hook up the storage that
>> they just created?****
>>
>>  ****
>>
>> I'm a bit familiar with how OpenStack does this and it seems that it only
>> has its storage plug-ins create a volume (LUN, whatever) and then the
>> framework handles the process of dealing with the hypervisor in question to
>> hook up the storage.****
>>
>>  ****
>>
>> It seems like otherwise we'd need to create a utility for all storage
>> plug-ins to share otherwise they'd be duplicating efforts in talking to
>> hypervisors.****
>>
>>  ****
>>
>> What do you think?****
>>
>>  ****
>>
>> On Thu, Mar 21, 2013 at 7:52 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com> wrote:****
>>
>> Hi Edison,****
>>
>>  ****
>>
>> I believe I understand the requirements for the plug-in better now.****
>>
>>  ****
>>
>> It sounds like the flow will be as such:****
>>
>>  ****
>>
>> * The user executes a Compute or Disk Offering that is tied via a storage
>> tag to a Primary Storage that is associated with a plug-in.****
>>
>>  ****
>>
>> * The storage framework will ask the plug-in to create a volume.  The
>> plug-in will create a volume and hook the volume up to the appropriate
>> hypervisor.  For VMware, this means the plug-in will create a Datastore.
>>  For XenServer, this means the plug-in will create a Storage Repository.
>>  (So on and so forth for other hypervisors.)****
>>
>>  ****
>>
>> * The VM or data disk is then deployed to the hypervisor.****
>>
>>  ****
>>
>> Does that sound correct, Edison?****
>>
>>  ****
>>
>> Thanks!****
>>
>>  ****
>>
>> On Thu, Mar 21, 2013 at 5:44 PM, Edison Su <Edison.su@citrix.com> wrote:*
>> ***
>>
>>  ****
>>
>>  ****
>>
>> *From:* Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>> *Sent:* Thursday, March 21, 2013 4:18 PM
>> *To:* Edison Su
>> *Subject:* Re: Storage Subsystem 2.0 plugin docs****
>>
>>  ****
>>
>> Hi Edison,****
>>
>>  ****
>>
>> I wanted to dive into these comments a bit more:****
>>
>>  ****
>>
>> [Edison] plugin’s driver->createasync will be called when mgt server want
>> to create a volume on the storage. In the driver’s implementation, it can
>> directly call storage box’s api, or send a command to hypervisor host, then
>> call storage box’s api to create an iscsi.****
>>
>> Then create a datastore(for vmware), SR(for xenserver), or storage
>> pool(for KVM) on hypervisor host, based on the iscsi iqn.****
>>
>> If the volume is created from a template(for root disk), need to find a
>> way to import that template(which is nfs based currently, it will be just a
>> plain http url the future) into the root disk.****
>>
>> The part about creating a datastore or a storage repository...is that
>> something the plug-in will be responsible for doing or will the storage
>> framework cover that piece?  I'm thinking the storage framework will since
>> all sorts of plug-ins would seem to need that ability (to have their
>> storage hooked up to a datastore or a storage repository).****
>>
>>  ****
>>
>> [Edison] It’s a specific requirement for per volume per LUN case, and
>> specific for certain hypervisors(seems KVM doesn’t need to create a storage
>> pool when using iscsi LUN), so the storage framework will not deal with it
>> right now.****
>>
>>  ****
>>
>>  ****
>>
>> Thanks for your time, Edison! :)****
>>
>>  ****
>>
>> On Thu, Mar 21, 2013 at 4:45 PM, Edison Su <Edison.su@citrix.com> wrote:*
>> ***
>>
>> Feedback/comments are appreciated, need to know your input from storage
>> vendor point of view.****
>>
>>  ****
>>
>> *From:* Vladimir Popovski [mailto:vladimir@zadarastorage.com]
>> *Sent:* Thursday, March 21, 2013 11:52 AM
>> *To:* Edison Su; cloudstack****
>>
>>
>> *Cc:* mike.tutkowski@solidfire.com
>> *Subject:* RE: Storage Subsystem 2.0 plugin docs****
>>
>>  ****
>>
>> Hi Edison,****
>>
>>  ****
>>
>> Thank you for the reply. We will check it out.****
>>
>>  ****
>>
>> Regards,****
>>
>> -Vladimir****
>>
>>  ****
>>
>>  ****
>>
>> *From:* Edison Su [mailto:Edison.su@citrix.com]
>> *Sent:* Thursday, March 21, 2013 11:36 AM
>> *To:* 'Vladimir Popovski'; cloudstack
>> *Cc:* mike.tutkowski@solidfire.com
>> *Subject:* RE: Storage Subsystem 2.0 plugin docs****
>>
>>  ****
>>
>>  ****
>>
>>  ****
>>
>> *From:* Vladimir Popovski [mailto:vladimir@zadarastorage.com<vladimir@zadarastorage.com>]
>>
>> *Sent:* Wednesday, March 20, 2013 9:05 AM
>> *To:* cloudstack
>> *Cc:* mike.tutkowski@solidfire.com; Edison Su
>> *Subject:* Storage Subsystem 2.0 plugin docs****
>>
>>  ****
>>
>> Hi All,****
>>
>>  ****
>>
>> Thank you for a great work on CloudStack! We are interested in
>> integrating CS with our storage system and started to look at your
>> documentation and storage-related code. I see that Mike from SolidFire
>> started working on something similar some time ago and Edison even created
>> an empty plugin for it (in Nov’12?).****
>>
>>  ****
>>
>> We have couple of questions related to that:****
>>
>> -          Is there any documentation about plugins (except of
>> https://cwiki.apache.org/CLOUDSTACK/storage-subsystem-20.html)****
>>
>> [Edison] There are not much docs about the plugins other than the above
>> link.  See below.****
>>
>> -          Are there any exemplary plugins for primary & secondary
>> datastores? Was the SolidFire plugin ever finished?****
>>
>> [Edison] yesterday, I checked in some code to separate existing
>> cloudstack storage code into a standalone maven project, called:
>> cloud-plugin-storage-volume-default, which can give you an example how a
>> storage plugin will look like.****
>>
>> -          How to activate a new plugin and use it (at least through
>> CLIs/APIs)****
>>
>> [Edison] First, put a bean configuration in client/tomcatconf/
>> componentContext.xml.in for your plugin provider class, like:****
>>
>> <bean id="ClassicalPrimaryDataStoreProvider"
>> class="org.apache.cloudstack.storage.datastore.provider.CloudStackPrimaryDataStoreProviderImpl">
>> ****
>>
>>   </bean>****
>>
>> Second, when adding a data store into cloudstack, with an extra parameter
>> in createstoragepoolcmd: provider=your-provider-name,
>> liststorageproviderscmd can list all the registered providers in mgt server.
>> ****
>>
>>  ****
>>
>>  ****
>>
>> -          How to integrate it with the UI****
>>
>>  There is no UI part of example code for storage yet, the idea is to use
>> pluggable UI(
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/UI+Plugin+Tutorial),
>> for each storage provider may need a separate UI to add a storage. For
>> example, in adding primary storage ui, there will be a drop down list, show
>> all the registered providers, if user selects one of the drop down list,
>> then UI will pop up a diagram, based on providers’ pluggable ui, then user
>> can type whatever information needed for a storage(e.g. nfs server, nfs
>> path, if its nfs). At the end, UI will call createstoragepoolcmd to
>> register a storage into cloudstack.****
>>
>>  ****
>>
>> Thanks,****
>>
>> -Vladimir****
>>
>>  ****
>>
>>  ****
>>
>> -------****
>>
>> Vladimir Popovski****
>>
>> VP, Cloud Operations****
>>
>> Zadara Storage
>> (949) 677-2095****
>>
>> Vladimir@zadarastorage.com****
>>
>> www.zadarastorage.com****
>>
>>  ****
>>
>>  ****
>>
>>
>>
>> ****
>>
>>  ****
>>
>> --
>> *Mike Tutkowski*****
>>
>> *Senior CloudStack Developer, SolidFire Inc.*****
>>
>> e: mike.tutkowski@solidfire.com****
>>
>> o: 303.746.7302****
>>
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *™*****
>>
>>
>>
>> ****
>>
>>  ****
>>
>> --
>> *Mike Tutkowski*****
>>
>> *Senior CloudStack Developer, SolidFire Inc.*****
>>
>> e: mike.tutkowski@solidfire.com****
>>
>> o: 303.746.7302****
>>
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *™*****
>>
>>
>>
>> ****
>>
>>  ****
>>
>> --
>> *Mike Tutkowski*****
>>
>> *Senior CloudStack Developer, SolidFire Inc.*****
>>
>> e: mike.tutkowski@solidfire.com****
>>
>> o: 303.746.7302****
>>
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *™*****
>>
>>
>>
>> ****
>>
>>  ****
>>
>> --
>> *Mike Tutkowski*****
>>
>> *Senior CloudStack Developer, SolidFire Inc.*****
>>
>> e: mike.tutkowski@solidfire.com****
>>
>> o: 303.746.7302****
>>
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *™*****
>>
>>
>>
>> ****
>>
>>  ****
>>
>> --
>> *Mike Tutkowski*****
>>
>> *Senior CloudStack Developer, SolidFire Inc.*****
>>
>> e: mike.tutkowski@solidfire.com****
>>
>> o: 303.746.7302****
>>
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *™*****
>>
>>
>>
>> ****
>>
>>  ****
>>
>> --
>> *Mike Tutkowski*****
>>
>> *Senior CloudStack Developer, SolidFire Inc.*****
>>
>> e: mike.tutkowski@solidfire.com****
>>
>> o: 303.746.7302****
>>
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *™*****
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *™*
>



-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message