cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Tutkowski <mike.tutkow...@solidfire.com>
Subject Re: [MERGE] disk_io_throttling to MASTER
Date Tue, 04 Jun 2013 21:28:32 GMT
"So, in essence, the SolidFire plugin introduces the notion of a managed
iSCSI device and provisioned IOPS."

Technically, the SolidFire plug-in just introduces the notion of
provisioned storage IOPS.

The storage framework that leverages the plug-in was incomplete, so I had
to try to add in the notion of a managed iSCSI device.

I appreciate all the time you've been spending on this. :) Do you have a
recommendation as to how we should accomplish what you're looking for?

Thanks!


On Tue, Jun 4, 2013 at 3:19 PM, John Burwell <jburwell@basho.com> wrote:

> Mike,
>
> So, in essence, the SolidFire plugin introduces the notion of a managed
> iSCSI device and provisioned IOPS.
>
> I want to see a separation of the management capabilities (i.e. can a
> device be managed/does an operator want it managed by CloudStack) from the
> storage protocol.  Ideally, we should end up with a semantic that will
> allow any type of storage device to be managed.  I also want to make
> progress on decoupling the storage types from the hypervisor definitions.
>
> Thanks,
> -John
>
> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski <mike.tutkowski@solidfire.com>
> wrote:
>
> > Hi John,
> >
> > No problem. Answers are below in red.
> >
> > Thanks!
> >
> >
> > On Tue, Jun 4, 2013 at 2:55 PM, John Burwell <jburwell@basho.com> wrote:
> >
> >> Mike,
> >>
> >> Could you please answer the following questions for me with regards to
> the
> >> operation of the SolidFire plugin:
> >>
> >> What is the cardinality between iSCSI LUNs and SAN volumes?
> >>
> > Each SAN volume is equivalent to a single LUN (LUN 0).
> > 1 SAN volume : 1 LUN
> >
> >> What is the cardinality between SAN Volumes and CloudStack Volumes?
> >>
> > 1 SAN volume : 1 CloudStack volume
> >
> >> Are the LUN(s) created by the management server or externally by the
> >> operator?
> >>
> > When used with the SolidFire plug-in, a SAN volume (same as a SAN LUN) is
> > created by the management server (via the plug-in) the first time the
> > CloudStack volume is attached to a hypervisor.
> >
> > If you don't want to use the SolidFire plug-in, but still want to use a
> > SolidFire volume (LUN), you can do this already today (prior to 4.2). The
> > admin manually creates the SAN volume and - in this case - multiple VMs
> and
> > data disks can share this SAN volume. While you can do this today, it is
> > not useful if you want to enforce storage QoS.
> >
> >> Are the SAN volumes by the management server or externally by the
> operator?
> >>
> > When the SolidFire plug-in is used, the SAN volumes are completely
> managed
> > by the management server (via the plug-in). There is no admin
> interaction.
> > This allows for a 1:1 mapping between a SAN volume and a CloudStack
> volume,
> > which is necessary for any storage vendor that supports true QoS.
> >
> >>
> >> I would like to clarify how these pieces are related and expected to
> >> operate.
> >>
> >> Thanks,
> >> -John
> >>
> >> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com>
> >> wrote:
> >>
> >>> "In particular, how do we ensure that multiple VMs with provisioned
> IOPS
> >>> won't be cut off by the underlying storage."
> >>>
> >>> In the storage QoS world, we need to map a single SAN volume (LUN) to a
> >>> single CloudStack volume. We cannot have multiple CloudStack volumes
> >>> sharing a single SAN volume and still guarantee QoS.
> >>>
> >>> If the user wants to have a single SAN volume house more than one
> >>> CloudStack volume, then can do that today without any of my plug-in
> code.
> >>>
> >>>
> >>> On Tue, Jun 4, 2013 at 1:43 PM, Mike Tutkowski <
> >> mike.tutkowski@solidfire.com
> >>>> wrote:
> >>>
> >>>> "The administrator will allocate a SAN volume for CloudStack's use
> onto
> >>>> which CloudStack volumes will be created."
> >>>>
> >>>> I think we crossed e-mails. :)
> >>>>
> >>>> Check out my recent e-mail on this.
> >>>>
> >>>>
> >>>> On Tue, Jun 4, 2013 at 1:41 PM, John Burwell <jburwell@basho.com>
> >> wrote:
> >>>>
> >>>>> Mike,
> >>>>>
> >>>>> You are coming to the part which concerns me -- concepts from the
> >>>>> hypervisor are leaking into storage layer.
> >>>>>
> >>>>> Thanks,
> >>>>> -John
> >>>>>
> >>>>> On Jun 4, 2013, at 3:35 PM, Mike Tutkowski <
> >> mike.tutkowski@solidfire.com>
> >>>>> wrote:
> >>>>>
> >>>>>> The weird part is that the iSCSI type is today only used (as far as
> I
> >>>>> know)
> >>>>>> in regards to XenServer (when you have not PreSetup an SR).
> >>>>>>
> >>>>>> If you want to use your iSCSI volume from VMware, it uses the vmfs
> >> type.
> >>>>>>
> >>>>>> If you want to use your iSCSI volume from KVM, it uses the
> >>>>> SharedMountPoint
> >>>>>> type.
> >>>>>>
> >>>>>> So, I suppose mine and Edison's thinking here was to make a new type
> >> of
> >>>>>> storage to describe this dynamic ability Edison added into the
> storage
> >>>>>> framework. Maybe it should be more specificy, though: Dynamic_iSCSI
> >>>>> versus,
> >>>>>> say, Dynamic_FC.
> >>>>>>
> >>>>>>
> >>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike Tutkowski <
> >>>>> mike.tutkowski@solidfire.com
> >>>>>>> wrote:
> >>>>>>
> >>>>>>> "The storage device itself shouldn't know or care that it is being
> >> used
> >>>>>>> for a Xen SR -- simply be able to answer questions about it is
> >>>>> storing."
> >>>>>>>
> >>>>>>> I see...so your concern here is that the SolidFire plug-in needs to
> >>>>> call
> >>>>>>> itself "Dynamic" storage so that the hypervisor logic knows to
> treat
> >>>>> it as
> >>>>>>> such.
> >>>>>>>
> >>>>>>> I'm totally open to removing that constraint and just calling it
> >> iSCSI
> >>>>> or
> >>>>>>> whatever. We would just need a way for the hypervisor attach logic
> to
> >>>>>>> detect this new requirement and perform the necessary activities.
> >>>>>>>
> >>>>>>>
> >>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John Burwell <jburwell@basho.com>
> >>>>> wrote:
> >>>>>>>
> >>>>>>>> Mike,
> >>>>>>>>
> >>>>>>>> See my responses in-line.
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> -John
> >>>>>>>>
> >>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike Tutkowski <
> >>>>> mike.tutkowski@solidfire.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> I'm trying to picture this:
> >>>>>>>>>
> >>>>>>>>> "Finally, while CloudStack may be able to manage a device, an
> >>>>> operator
> >>>>>>>> may
> >>>>>>>>> chose to leave it unmanaged by CloudStack (e.g. the device is
> >> shared
> >>>>> by
> >>>>>>>>> many services, and the operator has chosen to dedicate only a
> >> portion
> >>>>>>>> of it
> >>>>>>>>> to CloudStack).  Does my reasoning make sense?"
> >>>>>>>>>
> >>>>>>>>> I guess I'm not sure how creating a SAN volume via the plug-in
> >>>>> (before
> >>>>>>>> an
> >>>>>>>>> attach request to the hypervisor) would work unless the
> hypervisor
> >>>>>>>> consumes
> >>>>>>>>> the SAN volume in the form of, say, an SR.
> >>>>>>>>
> >>>>>>>> My thinking is that, independent of CloudStack, an operator
> >> allocates
> >>>>> a
> >>>>>>>> chunk of  a SAN to CloudStack, and exposes it through a LUN.  They
> >>>>> simply
> >>>>>>>> want to turn control of that LUN over to CloudStack, but not allow
> >>>>>>>> CloudStack to allocate anymore LUNs.
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>> As the attach logic stands prior to my changes, we would be
> passing
> >>>>> in a
> >>>>>>>>> SAN volume that does not have the necessary hypervisor support
> >> (like
> >>>>> an
> >>>>>>>> SR)
> >>>>>>>>> and the logic will fail.
> >>>>>>>>>
> >>>>>>>>> Are you thinking we should maybe have the storage framework
> itself
> >>>>>>>> detect
> >>>>>>>>> that such a SAN volume needs support from the hypervisor side and
> >>>>> have
> >>>>>>>> it
> >>>>>>>>> call into the agent code specifically to create the SR before the
> >>>>> attach
> >>>>>>>>> logic runs in the agent code?
> >>>>>>>>
> >>>>>>>> I think the hypervisor management plugin should have a rich enough
> >>>>>>>> interface to storage to determine available for volume storage.
>  For
> >>>>> Xen,
> >>>>>>>> this interface would allow the interrogation of the device to
> >>>>> determine the
> >>>>>>>> SR is present.   The storage device itself shouldn't know or care
> >>>>> that it
> >>>>>>>> is being used for a Xen SR -- simply be able to answer questions
> >>>>> about it
> >>>>>>>> is storing.
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike Tutkowski <
> >>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> So, the flow is as follows:
> >>>>>>>>>>
> >>>>>>>>>> * The admin registers the SolidFire driver (which is a type of
> >>>>>>>> so-called
> >>>>>>>>>> Dynamic storage). Once this is done, a new Primary Storage shows
> >> up
> >>>>> in
> >>>>>>>> the
> >>>>>>>>>> applicable zone.
> >>>>>>>>>>
> >>>>>>>>>> * The admin creates a Disk Offering that references the storage
> >> tag
> >>>>> of
> >>>>>>>> the
> >>>>>>>>>> newly created Primary Storage.
> >>>>>>>>>>
> >>>>>>>>>> * The end user creates a CloudStack volume. This leads to a new
> >> row
> >>>>> in
> >>>>>>>> the
> >>>>>>>>>> cloud.volumes table.
> >>>>>>>>>>
> >>>>>>>>>> * The end user attaches the CloudStack volume to a VM (attach
> >> disk).
> >>>>>>>> This
> >>>>>>>>>> leads to the storage framework calling the plug-in to create a
> new
> >>>>>>>> volume
> >>>>>>>>>> on its storage system (in my case, a SAN). The plug-in also
> >> updates
> >>>>> the
> >>>>>>>>>> cloud.volumes row with applicable data (like the IQN of the SAN
> >>>>>>>> volume).
> >>>>>>>>>> This plug-in code is only invoked if the CloudStack volume is in
> >> the
> >>>>>>>>>> 'Allocated' state. After the attach, the volume will be in the
> >>>>> 'Ready'
> >>>>>>>>>> state (even after a detach disk) and the plug-in code will not
> be
> >>>>>>>> called
> >>>>>>>>>> again to create this SAN volume.
> >>>>>>>>>>
> >>>>>>>>>> * The hypervisor-attach logic is run and detects the CloudStack
> >>>>> volume
> >>>>>>>> to
> >>>>>>>>>> attach needs "assistance" in the form of a hypervisor data
> >> structure
> >>>>>>>> (ex.
> >>>>>>>>>> an SR on XenServer).
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM, Mike Tutkowski <
> >>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> "To ensure that we are in sync on terminology, volume, in these
> >>>>>>>>>>> definitions, refers to the physical allocation on the device,
> >>>>>>>> correct?"
> >>>>>>>>>>>
> >>>>>>>>>>> Yes...when I say 'volume', I try to mean 'SAN volume'.
> >>>>>>>>>>>
> >>>>>>>>>>> To refer to the 'volume' the end user can make in CloudStack, I
> >>>>> try to
> >>>>>>>>>>> use 'CloudStack volume'.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM, Mike Tutkowski <
> >>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>
> >>>>>>>>>>>> What you say here may very well make sense, but I'm having a
> >> hard
> >>>>>>>> time
> >>>>>>>>>>>> envisioning it.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Perhaps we should draw Edison in on this conversation as he
> was
> >>>>> the
> >>>>>>>>>>>> initial person to suggest the approach I took.
> >>>>>>>>>>>>
> >>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM, John Burwell <
> >> jburwell@basho.com
> >>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> It feels like we are combining two distinct concepts --
> storage
> >>>>>>>> device
> >>>>>>>>>>>>> management and storage protocols.  In both cases, we are
> >>>>>>>> communicating with
> >>>>>>>>>>>>> ISCSI, but one allows the system to create/delete volumes
> >>>>> (Dynamic)
> >>>>>>>> on the
> >>>>>>>>>>>>> device while the other requires the volume to be volume to be
> >>>>>>>> managed
> >>>>>>>>>>>>> outside of the CloudStack context.  To ensure that we are in
> >>>>> sync on
> >>>>>>>>>>>>> terminology, volume, in these definitions, refers to the
> >> physical
> >>>>>>>>>>>>> allocation on the device, correct?  Minimally, we must be
> able
> >> to
> >>>>>>>>>>>>> communicate with a storage device to move bits from one place
> >> to
> >>>>>>>> another,
> >>>>>>>>>>>>> read bits, delete bits, etc.  Optionally, a storage device
> may
> >> be
> >>>>>>>> able to
> >>>>>>>>>>>>> managed by CloudStack.  Therefore, we can have a unmanaged
> >> iSCSI
> >>>>>>>> device
> >>>>>>>>>>>>> onto which we store a Xen SR, and we can have a managed
> >> SolidFire
> >>>>>>>> iSCSI
> >>>>>>>>>>>>> device on which CloudStack is capable of allocating LUNs and
> >>>>> storing
> >>>>>>>>>>>>> volumes.  Finally, while CloudStack may be able to manage a
> >>>>> device,
> >>>>>>>> an
> >>>>>>>>>>>>> operator may chose to leave it unmanaged by CloudStack (e.g.
> >> the
> >>>>>>>> device is
> >>>>>>>>>>>>> shared by many services, and the operator has chosen to
> >> dedicate
> >>>>>>>> only a
> >>>>>>>>>>>>> portion of it to CloudStack).  Does my reasoning make sense?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Assuming my thoughts above are reasonable, it seems
> appropriate
> >>>>> to
> >>>>>>>>>>>>> strip the management concerns from StoragePoolType, add the
> >>>>> notion
> >>>>>>>> of a
> >>>>>>>>>>>>> storage device with an attached driver that indicates whether
> >> or
> >>>>>>>> not is
> >>>>>>>>>>>>> managed by CloudStack, and establish a abstraction
> >> representing a
> >>>>>>>> physical
> >>>>>>>>>>>>> allocation on a device separate that is associated with a
> >> volume.
> >>>>>>>> With
> >>>>>>>>>>>>> these notions in place, hypervisor drivers can declare which
> >>>>>>>> protocols they
> >>>>>>>>>>>>> support and when they encounter a device managed by
> CloudStack,
> >>>>>>>> utilize the
> >>>>>>>>>>>>> management operations exposed by the driver to automate
> >>>>> allocation.
> >>>>>>>> If
> >>>>>>>>>>>>> these thoughts/concepts make sense, then we can sit down and
> >>>>> drill
> >>>>>>>> down to
> >>>>>>>>>>>>> a more detailed design.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>> -John
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM, Mike Tutkowski <
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Here is the difference between the current iSCSI type and
> the
> >>>>>>>> Dynamic
> >>>>>>>>>>>>> type:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> iSCSI type: The admin has to go in and create a Primary
> >> Storage
> >>>>>>>> based
> >>>>>>>>>>>>> on
> >>>>>>>>>>>>>> the iSCSI type. At this point in time, the iSCSI volume must
> >>>>> exist
> >>>>>>>> on
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>> storage system (it is pre-allocated). Future CloudStack
> >> volumes
> >>>>> are
> >>>>>>>>>>>>> created
> >>>>>>>>>>>>>> as VDIs on the SR that was created behind the scenes.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Dynamic type: The admin has to go in and create Primary
> >> Storage
> >>>>>>>> based
> >>>>>>>>>>>>> on a
> >>>>>>>>>>>>>> plug-in that will create and delete volumes on its storage
> >>>>> system
> >>>>>>>>>>>>>> dynamically (as is enabled via the storage framework). When
> a
> >>>>> user
> >>>>>>>>>>>>> wants to
> >>>>>>>>>>>>>> attach a CloudStack volume that was created, the framework
> >> tells
> >>>>>>>> the
> >>>>>>>>>>>>>> plug-in to create a new volume. After this is done, the
> attach
> >>>>>>>> logic
> >>>>>>>>>>>>> for
> >>>>>>>>>>>>>> the hypervisor in question is called. No hypervisor data
> >>>>> structure
> >>>>>>>>>>>>> exists
> >>>>>>>>>>>>>> at this point because the volume was just created. The
> >>>>> hypervisor
> >>>>>>>> data
> >>>>>>>>>>>>>> structure must be created.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski <
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> These are new terms, so I should probably have defined them
> >> up
> >>>>>>>> front
> >>>>>>>>>>>>> for
> >>>>>>>>>>>>>>> you. :)
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Static storage: Storage that is pre-allocated (ex. an admin
> >>>>>>>> creates a
> >>>>>>>>>>>>>>> volume on a SAN), then a hypervisor data structure is
> created
> >>>>> to
> >>>>>>>>>>>>> consume
> >>>>>>>>>>>>>>> the storage (ex. XenServer SR), then that hypervisor data
> >>>>>>>> structure
> >>>>>>>>>>>>> is
> >>>>>>>>>>>>>>> consumed by CloudStack. Disks (VDI) are later placed on
> this
> >>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>> data structure as needed. In these cases, the attach logic
> >>>>> assumes
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>> hypervisor data structure is already in place and simply
> >>>>> attaches
> >>>>>>>>>>>>> the VDI
> >>>>>>>>>>>>>>> on the hypervisor data structure to the VM in question.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Dynamic storage: Storage that is not pre-allocated. Instead
> >> of
> >>>>>>>>>>>>>>> pre-existent storage, this could be a SAN (not a volume on
> a
> >>>>> SAN,
> >>>>>>>>>>>>> but the
> >>>>>>>>>>>>>>> SAN itself). The hypervisor data structure must be created
> >>>>> when an
> >>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>> volume is performed because these types of volumes have not
> >>>>> been
> >>>>>>>>>>>>> pre-hooked
> >>>>>>>>>>>>>>> up to such a hypervisor data structure by an admin. Once
> the
> >>>>>>>> attach
> >>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>> creates, say, an SR on XenServer for this volume, it
> attaches
> >>>>> the
> >>>>>>>>>>>>> one and
> >>>>>>>>>>>>>>> only VDI within the SR to the VM in question.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell <
> >>>>> jburwell@basho.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> The current implementation of the Dynamic type attach
> >> behavior
> >>>>>>>>>>>>> works in
> >>>>>>>>>>>>>>>> terms of Xen ISCSI which why I ask about the difference.
> >>>>> Another
> >>>>>>>>>>>>> way to
> >>>>>>>>>>>>>>>> ask the question -- what is the definition of a Dynamic
> >>>>> storage
> >>>>>>>>>>>>> pool type?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski <
> >>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> As far as I know, the iSCSI type is uniquely used by
> >>>>> XenServer
> >>>>>>>>>>>>> when you
> >>>>>>>>>>>>>>>>> want to set up Primary Storage that is directly based on
> an
> >>>>>>>> iSCSI
> >>>>>>>>>>>>>>>> target.
> >>>>>>>>>>>>>>>>> This allows you to skip the step of going to the
> hypervisor
> >>>>> and
> >>>>>>>>>>>>>>>> creating a
> >>>>>>>>>>>>>>>>> storage repository based on that iSCSI target as
> CloudStack
> >>>>> does
> >>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>> part
> >>>>>>>>>>>>>>>>> for you. I think this is only supported for XenServer.
> For
> >>>>> all
> >>>>>>>>>>>>> other
> >>>>>>>>>>>>>>>>> hypervisors, you must first go to the hypervisor and
> >> perform
> >>>>>>>> this
> >>>>>>>>>>>>> step
> >>>>>>>>>>>>>>>>> manually.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I don't really know what RBD is.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell <
> >>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Reading through the code, what is the difference between
> >> the
> >>>>>>>>>>>>> ISCSI and
> >>>>>>>>>>>>>>>>>> Dynamic types?  Why isn't RBD considered Dynamic?
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> This new type of storage is defined in the
> >>>>>>>>>>>>> Storage.StoragePoolType
> >>>>>>>>>>>>>>>> class
> >>>>>>>>>>>>>>>>>>> (called Dynamic):
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> public static enum StoragePoolType {
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>  Filesystem(false), // local directory
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>  NetworkFilesystem(true), // NFS or CIFS
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>  IscsiLUN(true), // shared LUN, with a clusterfs
> overlay
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>  Iscsi(true), // for e.g., ZFS Comstar
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>  ISO(false), // for iso image
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>  LVM(false), // XenServer local LVM SR
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>  CLVM(true),
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>  RBD(true),
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>  SharedMountPoint(true),
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>  VMFS(true), // VMware VMFS storage
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>  PreSetup(true), // for XenServer, Storage Pool is set
> >> up
> >>>>>>>> by
> >>>>>>>>>>>>>>>>>>> customers.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>  EXT(false), // XenServer local EXT SR
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>  OCFS2(true),
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>  Dynamic(true); // dynamic, zone-wide storage (ex.
> >>>>>>>> SolidFire)
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>  boolean shared;
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>  StoragePoolType(boolean shared) {
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>      this.shared = shared;
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>  }
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>  public boolean isShared() {
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>      return shared;
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>  }
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> For example, let's say another storage company wants
> to
> >>>>>>>>>>>>> implement a
> >>>>>>>>>>>>>>>>>>>> plug-in to leverage its Quality of Service feature. It
> >>>>> would
> >>>>>>>> be
> >>>>>>>>>>>>>>>> dynamic,
> >>>>>>>>>>>>>>>>>>>> zone-wide storage, as well. They would need only
> >>>>> implement a
> >>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>> plug-in as I've made the necessary changes to the
> >>>>>>>>>>>>> hypervisor-attach
> >>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>> to support their plug-in.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply the XenServer code is SolidFire
> >>>>> specific.
> >>>>>>>>>>>>> It is
> >>>>>>>>>>>>>>>> not.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> The XenServer attach logic is now aware of dynamic,
> >>>>>>>> zone-wide
> >>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>> (and SolidFire is an implementation of this kind of
> >>>>>>>> storage).
> >>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>> kind of
> >>>>>>>>>>>>>>>>>>>>> storage is new to 4.2 with Edison's storage framework
> >>>>>>>> changes.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Edison created a new framework that supported the
> >>>>> creation
> >>>>>>>> and
> >>>>>>>>>>>>>>>> deletion
> >>>>>>>>>>>>>>>>>>>>> of volumes dynamically. However, when I visited with
> >> him
> >>>>> in
> >>>>>>>>>>>>> Portland
> >>>>>>>>>>>>>>>>>> back
> >>>>>>>>>>>>>>>>>>>>> in April, we realized that it was not complete. We
> >>>>> realized
> >>>>>>>>>>>>> there
> >>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>> nothing CloudStack could do with these volumes unless
> >> the
> >>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>> changed to recognize this new type of storage and
> >> create
> >>>>> the
> >>>>>>>>>>>>>>>>>> appropriate
> >>>>>>>>>>>>>>>>>>>>> hypervisor data structure.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell <
> >>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> It is generally odd to me that any operation in the
> >>>>> Storage
> >>>>>>>>>>>>> layer
> >>>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>> understand or care about details.  I expect to see
> the
> >>>>>>>> Storage
> >>>>>>>>>>>>>>>>>> services
> >>>>>>>>>>>>>>>>>>>>>> expose a set of operations that can be
> composed/driven
> >>>>> by
> >>>>>>>> the
> >>>>>>>>>>>>>>>>>> Hypervisor
> >>>>>>>>>>>>>>>>>>>>>> implementations to allocate space/create structures
> >> per
> >>>>>>>> their
> >>>>>>>>>>>>>>>> needs.
> >>>>>>>>>>>>>>>>>> If
> >>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>> don't invert this dependency, we are going to end
> >> with a
> >>>>>>>>>>>>> massive
> >>>>>>>>>>>>>>>>>> n-to-n
> >>>>>>>>>>>>>>>>>>>>>> problem that will make the system increasingly
> >>>>> difficult to
> >>>>>>>>>>>>>>>> maintain
> >>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>> enhance.  Am I understanding that the Xen specific
> >>>>>>>> SolidFire
> >>>>>>>>>>>>> code
> >>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>> located in the CitrixResourceBase class?
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> To delve into this in a bit more detail:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and aside from one setup method for
> >>>>>>>> XenServer,
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>>>>>>> had
> >>>>>>>>>>>>>>>>>>>>>>> to first create a volume on the storage system,
> then
> >> go
> >>>>>>>> into
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>> to set up a data structure to make use of the
> volume
> >>>>> (ex.
> >>>>>>>> a
> >>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>> repository on XenServer or a datastore on ESX(i)).
> >> VMs
> >>>>> and
> >>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>> disks
> >>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>> shared this storage system's volume.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> With Edison's new storage framework, storage need
> no
> >>>>>>>> longer
> >>>>>>>>>>>>> be so
> >>>>>>>>>>>>>>>>>>>>>> static
> >>>>>>>>>>>>>>>>>>>>>>> and you can easily create a 1:1 relationship
> between
> >> a
> >>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>> system's
> >>>>>>>>>>>>>>>>>>>>>>> volume and the VM's data disk (necessary for
> storage
> >>>>>>>> Quality
> >>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>> Service).
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> You can now write a plug-in that is called to
> >>>>> dynamically
> >>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>> delete
> >>>>>>>>>>>>>>>>>>>>>>> volumes as needed.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> The problem that the storage framework did not
> >> address
> >>>>> is
> >>>>>>>> in
> >>>>>>>>>>>>>>>> creating
> >>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>> deleting the hypervisor-specific data structure
> when
> >>>>>>>>>>>>> performing an
> >>>>>>>>>>>>>>>>>>>>>>> attach/detach.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> That being the case, I've been enhancing it to do
> so.
> >>>>> I've
> >>>>>>>>>>>>> got
> >>>>>>>>>>>>>>>>>>>>>> XenServer
> >>>>>>>>>>>>>>>>>>>>>>> worked out and submitted. I've got ESX(i) in my
> >> sandbox
> >>>>>>>> and
> >>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>> submit
> >>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>> if we extend the 4.2 freeze date.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Does that help a bit? :)
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> The storage plug-in - by itself - is hypervisor
> >>>>> agnostic.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> The issue is with the volume-attach logic (in the
> >>>>> agent
> >>>>>>>>>>>>> code).
> >>>>>>>>>>>>>>>> The
> >>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>> framework calls into the plug-in to have it
> create a
> >>>>>>>> volume
> >>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>> needed,
> >>>>>>>>>>>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>>>>>>>>>> when the time comes to attach the volume to a
> >>>>> hypervisor,
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>>>> has to be smart enough to recognize it's being
> >>>>> invoked on
> >>>>>>>>>>>>>>>> zone-wide
> >>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>> (where the volume has just been created) and
> create,
> >>>>>>>> say, a
> >>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>> repository (for XenServer) or a datastore (for
> >>>>> VMware) to
> >>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>>>>>>> of the
> >>>>>>>>>>>>>>>>>>>>>>>> volume that was just created.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> I've been spending most of my time recently making
> >> the
> >>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>> logic
> >>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>> in the agent code.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Does that clear it up?
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John Burwell <
> >>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why the the storage driver is
> >>>>> hypervisor
> >>>>>>>>>>>>>>>> specific?
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Yes, ultimately I would like to support all
> >>>>> hypervisors
> >>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>>>>>>>>>> supports. I think I'm just out of time for 4.2
> to
> >>>>> get
> >>>>>>>> KVM
> >>>>>>>>>>>>> in.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Right now this plug-in supports XenServer.
> >>>>> Depending on
> >>>>>>>>>>>>> what
> >>>>>>>>>>>>>>>> we do
> >>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2 feature freeze, I have it working
> >> for
> >>>>>>>>>>>>> VMware in
> >>>>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>>>>>>> sandbox,
> >>>>>>>>>>>>>>>>>>>>>>>>>> as well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to be clear, this is all in regards
> to
> >>>>> Disk
> >>>>>>>>>>>>>>>> Offerings.
> >>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>> plan to
> >>>>>>>>>>>>>>>>>>>>>>>>>> support Compute Offerings post 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison
> >>>>> Damage
> >>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any plan on supporting KVM in the
> patch
> >>>>> cycle
> >>>>>>>>>>>>> post
> >>>>>>>>>>>>>>>> 4.2?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original Message -----
> >>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike Tutkowski" <
> >>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to
> MASTER
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on merging Wei's feature first, then
> >> mine.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature is for KVM only, then it is a
> non
> >>>>> issue
> >>>>>>>>>>>>> as I
> >>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <
> >>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> For the billing, as no one works on billing
> now,
> >>>>>>>> users
> >>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>> calculate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing by themselves. They can get the
> >>>>>>>>>>>>> service_offering
> >>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering of a VMs and volumes for
> >>>>> calculation.
> >>>>>>>> Of
> >>>>>>>>>>>>> course
> >>>>>>>>>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>>>>>>>>>> better
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user the exact limitation value of
> >>>>> individual
> >>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> rate limitation for nics as well. I can work
> on
> >> it
> >>>>>>>>>>>>> later. Do
> >>>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>>>>>>>> think it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of I/O throttling?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my misunstand the second the question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with what you said about the two
> features.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <
> >>>>>>>>>>>>> ustcweizhou@gmail.com
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John, Mike
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope Mike's aswer helps you. I am trying
> to
> >>>>>>>> adding
> >>>>>>>>>>>>> more.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I think billing should depend on IO
> >>>>> statistics
> >>>>>>>>>>>>> rather
> >>>>>>>>>>>>>>>> than
> >>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation. Please review disk_io_stat if
> you
> >>>>> have
> >>>>>>>>>>>>> time.
> >>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the IO statistics including bytes/iops
> >>>>>>>> read/write
> >>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>>>>>>>> individual
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual machine.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by the AWS model, customers are billed
> >> more
> >>>>>>>> for
> >>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned IOPS, as well as, for those
> >>>>> operations (
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/).  I would
> imagine
> >>>>> our
> >>>>>>>>>>>>> users
> >>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>> like
> >>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to employ similar cost models.  Could
> an
> >>>>>>>>>>>>> operator
> >>>>>>>>>>>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>>>>>>>>>>>>>> such a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing model in the current patch?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime change? KVM
> >>>>> supports
> >>>>>>>>>>>>> setting
> >>>>>>>>>>>>>>>>>>>>>> IOPS/BPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation for a running virtual machine
> >> through
> >>>>>>>>>>>>> command
> >>>>>>>>>>>>>>>> line.
> >>>>>>>>>>>>>>>>>>>>>>>>>>> However,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack does not support changing the
> >>>>> parameters
> >>>>>>>>>>>>> of a
> >>>>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> offering
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (computer offering or disk offering).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at the Java interface level.  I
> >> apologize
> >>>>>>>> for
> >>>>>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>>>>>> unclear.
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more generalize allocation algorithms
> with a
> >>>>> set
> >>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>> interfaces
> >>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe the service guarantees provided by a
> >>>>>>>> resource?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a good question. Maybe it is
> better
> >> to
> >>>>>>>>>>>>> commit
> >>>>>>>>>>>>>>>> Mike's
> >>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling as Mike needs to consider
> >> the
> >>>>>>>>>>>>>>>> limitation in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I think.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will expand on my thoughts in a later
> >> response
> >>>>> to
> >>>>>>>>>>>>> Mike
> >>>>>>>>>>>>>>>>>>>>>> regarding
> >>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch points between these two features.  I
> >> think
> >>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need to be merged before SolidFire, but
> I
> >>>>> think
> >>>>>>>>>>>>> we need
> >>>>>>>>>>>>>>>>>>>>>> closer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> coordination between the branches (possibly
> >> have
> >>>>>>>> have
> >>>>>>>>>>>>>>>> solidfire
> >>>>>>>>>>>>>>>>>>>>>>> track
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling) to coordinate on this
> >> issue.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell <jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The things I want to understand are the
> >>>>> following:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is there value in capturing IOPS
> policies
> >> be
> >>>>>>>>>>>>> captured
> >>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>> common
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data model (e.g. for billing/usage
> purposes,
> >>>>>>>>>>>>> expressing
> >>>>>>>>>>>>>>>>>>>>>>> offerings).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Should there be a common interface model
> >> for
> >>>>>>>>>>>>> reasoning
> >>>>>>>>>>>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>>>>>>>>>>>> IOP
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioning at runtime?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How are conflicting provisioned IOPS
> >>>>>>>>>>>>> configurations
> >>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor and storage device reconciled?
>  In
> >>>>>>>>>>>>> particular,
> >>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> where
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead to believe (and billed) for more
> IOPS
> >>>>>>>>>>>>> configured
> >>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>> a VM
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> than a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device has been configured to
> >> deliver.
> >>>>>>>>>>>>> Another
> >>>>>>>>>>>>>>>>>>>>>> scenario
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> could a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consistent configuration between a VM and a
> >>>>>>>> storage
> >>>>>>>>>>>>>>>> device at
> >>>>>>>>>>>>>>>>>>>>>>>>>>> creation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time, but a later modification to storage
> >>>>> device
> >>>>>>>>>>>>>>>> introduces
> >>>>>>>>>>>>>>>>>>>>>>> logical
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> inconsistency.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski
> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I believe Wei's feature deals with
> >> controlling
> >>>>> the
> >>>>>>>>>>>>> max
> >>>>>>>>>>>>>>>>>>>>>> number of
> >>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor side.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My feature is focused on controlling IOPS
> >> from
> >>>>> the
> >>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>> system
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> side.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope that helps. :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John
> Burwell
> >> <
> >>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My opinion is that no features should be
> >>>>> merged
> >>>>>>>>>>>>> until all
> >>>>>>>>>>>>>>>>>>>>>>>>>>> functional
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues have been resolved and it is ready
> to
> >>>>> turn
> >>>>>>>>>>>>> over to
> >>>>>>>>>>>>>>>>>>>>>> test.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Until
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total Ops vs discrete read/write ops issue
> >> is
> >>>>>>>>>>>>> addressed
> >>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> re-reviewed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, I don't think this criteria has been
> >>>>>>>>>>>>> satisfied.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, how does this work
> >> intersect/compliment
> >>>>> the
> >>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>> (
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/)?
>  As I
> >>>>>>>>>>>>> understand
> >>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also involves provisioned IOPS.  I would
> >> like
> >>>>> to
> >>>>>>>>>>>>> ensure
> >>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> have a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario where provisioned IOPS in KVM and
> >>>>>>>>>>>>> SolidFire are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> unnecessarily
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> incompatible.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <
> >>>>>>>>>>>>>>>> ustcweizhou@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I will change it next week.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander <
> wido@widodh.nl
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I have pushed the features into
> >>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If anyone object thems for technical
> reason
> >>>>> till
> >>>>>>>>>>>>> Monday,
> >>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> revert
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the sake of clarity I just want to
> >> mention
> >>>>>>>> again
> >>>>>>>>>>>>>>>> that we
> >>>>>>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps asap so that we
> >>>>> never
> >>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>> version
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only total IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You laid the groundwork for the I/O
> >> throttling
> >>>>>>>> and
> >>>>>>>>>>>>> that's
> >>>>>>>>>>>>>>>>>>>>>> great!
> >>>>>>>>>>>>>>>>>>>>>>>>> We
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> however prevent that we create legacy from
> >> day
> >>>>>>>> #1.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
> >> wido@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John Burwell
> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this enhancement must to discretely
> >>>>> support
> >>>>>>>>>>>>> read
> >>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>> write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see how it could be fixed later
> >> because
> >>>>> I
> >>>>>>>>>>>>> don't see
> >>>>>>>>>>>>>>>>>>>>>> how we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correctly
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split total IOPS into read and write.
> >>>>>>>> Therefore, we
> >>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>> stuck
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total unless/until we decided to break
> >>>>> backwards
> >>>>>>>>>>>>>>>>>>>>>> compatibility.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What Wei meant was merging it into master
> >> now
> >>>>> so
> >>>>>>>>>>>>> that it
> >>>>>>>>>>>>>>>>>>>>>> will go
> >>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch and add Read / Write IOps
> before
> >>>>> the
> >>>>>>>> 4.2
> >>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be released with Read and Write
> instead
> >>>>> of
> >>>>>>>>>>>>> Total
> >>>>>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is to make the May 31st feature
> freeze
> >>>>> date.
> >>>>>>>>>>>>> But if
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>> window
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see other threads) then it won't be
> >> necessary
> >>>>>>>> to do
> >>>>>>>>>>>>>>>> that.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also completely agree that there is no
> >>>>>>>> association
> >>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk I/O.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido den
> >>>>> Hollander <
> >>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks. Good question.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I  thought about at the beginning.
> Finally I
> >>>>>>>>>>>>> decided to
> >>>>>>>>>>>>>>>>>>>>>> ignore
> >>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference of read and write mainly
> because
> >>>>> the
> >>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>> throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> did
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care the difference of sent and received
> >>>>> bytes as
> >>>>>>>>>>>>> well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That reasoning seems odd. Networking and
> >> disk
> >>>>> I/O
> >>>>>>>>>>>>>>>> completely
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> different.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is much more expensive in most
> >>>>>>>> situations
> >>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bandwith.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Implementing it will be some copy-paste
> >> work.
> >>>>> It
> >>>>>>>>>>>>> could be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implemented in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few days. For the deadline of feature
> >> freeze,
> >>>>> I
> >>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that , if needed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It think it's a feature we can't miss. But
> >> if
> >>>>> it
> >>>>>>>>>>>>> goes
> >>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window we have to make sure we don't
> release
> >>>>> with
> >>>>>>>>>>>>> only
> >>>>>>>>>>>>>>>> total
> >>>>>>>>>>>>>>>>>>>>>>> IOps
> >>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse users.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander <
> >> wido@widodh.nl>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to merge disk_io_throttling
> >>>>> branch
> >>>>>>>> into
> >>>>>>>>>>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody object, I will merge into master
> >> in
> >>>>> 48
> >>>>>>>>>>>>> hours.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The purpose is :
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual machines are running on the same
> >>>>> storage
> >>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>> (local
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage or
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share strage). Because of the rate
> >> limitation
> >>>>> of
> >>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>>>>>> (such as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops), if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one VM has large disk operation, it may
> >> affect
> >>>>>>>> the
> >>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>>>>> performance
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other VMs running on the same storage
> >> device.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is neccesary to set the maximum rate
> and
> >>>>> limit
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>> disk
> >>>>>>>>>>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Looking at the code I see you make no
> >>>>> difference
> >>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu and libvirt support setting both a
> >>>>> different
> >>>>>>>>>>>>> rate
> >>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps which could benefit a lot of users.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's also strange, in the polling side you
> >>>>>>>> collect
> >>>>>>>>>>>>> both
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> Read
> >>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps, but on the throttling side you only
> go
> >>>>> for
> >>>>>>>> a
> >>>>>>>>>>>>> global
> >>>>>>>>>>>>>>>>>>>>>> value.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write IOps are usually much more expensive
> >>>>> then
> >>>>>>>> Read
> >>>>>>>>>>>>>>>> IOps,
> >>>>>>>>>>>>>>>>>>>>>> so it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a valid use-case where that an admin
> >>>>> would
> >>>>>>>> set
> >>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>> lower
> >>>>>>>>>>>>>>>>>>>>>>> value
> >>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps vs Read IOps.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since this only supports KVM at this
> point I
> >>>>>>>> think
> >>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value to at least have the mechanism in
> >> place
> >>>>> to
> >>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>>>>>> both,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this later would be a lot of work.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a hypervisor doesn't support setting
> >>>>> different
> >>>>>>>>>>>>> values
> >>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>> read
> >>>>>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write you can always sum both up and set
> >> that
> >>>>> as
> >>>>>>>> the
> >>>>>>>>>>>>>>>> total
> >>>>>>>>>>>>>>>>>>>>>>> limit.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why you implemented it
> this
> >>>>> way?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The feature includes:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in
> >>>>>>>> disk_offering,
> >>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>> global
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configuration)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) change the maximum rate of VMs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total bps and
> iops)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket:
> https://issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://
> >>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I will update later) :
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
> >>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>>>>>>>>>> https://cwiki.
> >>>>>>>>>>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>
> >>>>>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Merge check list :-
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did you check the branch's RAT execution
> >>>>>>>> success?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are there new dependencies introduced?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What automated testing (unit and
> >>>>> integration)
> >>>>>>>> is
> >>>>>>>>>>>>>>>> included
> >>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit tests are added.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What testing has been done to check for
> >>>>>>>> potential
> >>>>>>>>>>>>>>>>>>>>>> regressions?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on
> >>>>>>>> CloudStack
> >>>>>>>>>>>>> UI.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) VM operations, including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy,
> >> expunge.
> >>>>>>>>>>>>> migrate,
> >>>>>>>>>>>>>>>>>>>>>> restore
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) Volume operations, including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach, Detach
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To review the code, you can try
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff
> >>>>>>>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> f2e5591b710d04cc86815044f5823e******73a4a58944
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>
> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******
> >>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**
> >>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**
> >>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling<https://****
> >>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/****
> >>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> display/CLOUDSTACK/VM+Disk+IO+****Throttling<
> >>>>>>>>>>>>>>>> https://cwiki.
> >>>>>>>>>>>>>>>>>>>>>> **
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>
> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>
> >>>>>
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ht**tps://
> >>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-2071
> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071
> >>>>>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <**
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071
> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <h**ttps://
> >>>>>>>>>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -     VM Disk I/O Throttling)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *™*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *™*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>> *™*
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >
> >>>>>>>>>>>>>>>>>>>>>>>> *™*
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>> *™*
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>> *™*
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>> *™*
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>> cloud<
> http://solidfire.com/solution/overview/?video=play
> >>>
> >>>>>>>>>>>>>>>>>>> *™*
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
> >
> >>>>>>>>>>>>>>>>> *™*
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>> *™*
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> --
> >>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>> *™*
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> --
> >>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>> *™*
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>> *™*
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>> o: 303.746.7302
> >>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>> *™*
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> *Mike Tutkowski*
> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>> o: 303.746.7302
> >>>>>>>>> Advancing the way the world uses the
> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>> *™*
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> *Mike Tutkowski*
> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>> o: 303.746.7302
> >>>>>>> Advancing the way the world uses the cloud<
> >>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>> *™*
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> *Mike Tutkowski*
> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>> e: mike.tutkowski@solidfire.com
> >>>>>> o: 303.746.7302
> >>>>>> Advancing the way the world uses the
> >>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>> *™*
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>> --
> >>>> *Mike Tutkowski*
> >>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>> e: mike.tutkowski@solidfire.com
> >>>> o: 303.746.7302
> >>>> Advancing the way the world uses the cloud<
> >> http://solidfire.com/solution/overview/?video=play>
> >>>> *™*
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the
> >>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>> *™*
> >>
> >>
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *™*
>
>


-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message