Return-Path: X-Original-To: apmail-cloudstack-dev-archive@www.apache.org Delivered-To: apmail-cloudstack-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 9B2231084B for ; Tue, 4 Jun 2013 21:42:55 +0000 (UTC) Received: (qmail 39468 invoked by uid 500); 4 Jun 2013 21:42:55 -0000 Delivered-To: apmail-cloudstack-dev-archive@cloudstack.apache.org Received: (qmail 39431 invoked by uid 500); 4 Jun 2013 21:42:55 -0000 Mailing-List: contact dev-help@cloudstack.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cloudstack.apache.org Delivered-To: mailing list dev@cloudstack.apache.org Received: (qmail 39423 invoked by uid 99); 4 Jun 2013 21:42:55 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 04 Jun 2013 21:42:55 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of jburwell@basho.com designates 209.85.223.169 as permitted sender) Received: from [209.85.223.169] (HELO mail-ie0-f169.google.com) (209.85.223.169) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 04 Jun 2013 21:42:44 +0000 Received: by mail-ie0-f169.google.com with SMTP id 10so1721905ied.14 for ; Tue, 04 Jun 2013 14:42:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=references:from:mime-version:in-reply-to:date:message-id:subject:to :content-type:content-transfer-encoding:x-gm-message-state; bh=QmxjX8AQNua7ULhZe8ZpqgUyv86HWg6oZ9gPQENPeys=; b=QJK3RWyBj09Vhzuf6Da36KYURkBO1CqMGQDar13jOX7FC44znGlyUET+1iomfLqGGO CuD+PIK6yIWyuLhQZIXNfcSogJafa79qxyQhiKQkiPtRYix+PQDUU4lA0OIt+hu/4zL2 cvnSOyipLqebZDDyqXjG6phIDuKOgUXqCCR5vPxrTJ8+Hkhu/rA2YzWk0dH/WVWZmQ4I hvC/QDp1LGSHe3vj1zoS3E0OMVo6F/6AunLyogN7+Z98GSo/0zngdLQ/Od9F6HxDHIOm QYB8y3xeUdib28sfFT/JIQ4w3KjqjEh/6WsijzmlCOFURwuLgDRwMO0oG9BGGaZ2YqoB LO5g== X-Received: by 10.50.40.7 with SMTP id t7mr1890513igk.112.1370382143545; Tue, 04 Jun 2013 14:42:23 -0700 (PDT) References: <-6717161953645650340@unknownmsgid> <7966164390304114442@unknownmsgid> <1637190856.747092.1370279686542.JavaMail.root@bbits.ca> <93B5A79D-CDC1-4D3B-9742-16CE018454BA@basho.com> <390BD462-1751-4FFE-B237-CDD19D2DCEA7@basho.com> <2CDCC854-9144-48A2-ABA7-8D11B239BCA4@basho.com> <33D19C43-D344-462C-8E32-DF27E20E7D53@basho.com> From: John Burwell Mime-Version: 1.0 (1.0) In-Reply-To: Date: Tue, 4 Jun 2013 17:42:05 -0400 Message-ID: <9184425221219417149@unknownmsgid> Subject: Re: [MERGE] disk_io_throttling to MASTER To: "dev@cloudstack.apache.org" Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-Gm-Message-State: ALoCoQl/Df3iMq2MUAKaLUPZhPHvvGdFI1ui3YIjhLyVAOae2jiG2FEilAI0jaHicAcrgygF0wdU X-Virus-Checked: Checked by ClamAV on apache.org Mike, I am never at a loss for an opinion. I some thoughts, but want to confirm assumptions and ideas against the solidfire, disk_io_throttle, and object_store branches. I hope to collect them in a coherent form tomorrow (5 June 2013). Thanks, -John On Jun 4, 2013, at 5:29 PM, Mike Tutkowski w= rote: > "So, in essence, the SolidFire plugin introduces the notion of a managed > iSCSI device and provisioned IOPS." > > Technically, the SolidFire plug-in just introduces the notion of > provisioned storage IOPS. > > The storage framework that leverages the plug-in was incomplete, so I had > to try to add in the notion of a managed iSCSI device. > > I appreciate all the time you've been spending on this. :) Do you have a > recommendation as to how we should accomplish what you're looking for? > > Thanks! > > > On Tue, Jun 4, 2013 at 3:19 PM, John Burwell wrote: > >> Mike, >> >> So, in essence, the SolidFire plugin introduces the notion of a managed >> iSCSI device and provisioned IOPS. >> >> I want to see a separation of the management capabilities (i.e. can a >> device be managed/does an operator want it managed by CloudStack) from t= he >> storage protocol. Ideally, we should end up with a semantic that will >> allow any type of storage device to be managed. I also want to make >> progress on decoupling the storage types from the hypervisor definitions= . >> >> Thanks, >> -John >> >> On Jun 4, 2013, at 5:13 PM, Mike Tutkowski >> wrote: >> >>> Hi John, >>> >>> No problem. Answers are below in red. >>> >>> Thanks! >>> >>> >>> On Tue, Jun 4, 2013 at 2:55 PM, John Burwell wrote= : >>> >>>> Mike, >>>> >>>> Could you please answer the following questions for me with regards to >> the >>>> operation of the SolidFire plugin: >>>> >>>> What is the cardinality between iSCSI LUNs and SAN volumes? >>> Each SAN volume is equivalent to a single LUN (LUN 0). >>> 1 SAN volume : 1 LUN >>> >>>> What is the cardinality between SAN Volumes and CloudStack Volumes? >>> 1 SAN volume : 1 CloudStack volume >>> >>>> Are the LUN(s) created by the management server or externally by the >>>> operator? >>> When used with the SolidFire plug-in, a SAN volume (same as a SAN LUN) = is >>> created by the management server (via the plug-in) the first time the >>> CloudStack volume is attached to a hypervisor. >>> >>> If you don't want to use the SolidFire plug-in, but still want to use a >>> SolidFire volume (LUN), you can do this already today (prior to 4.2). T= he >>> admin manually creates the SAN volume and - in this case - multiple VMs >> and >>> data disks can share this SAN volume. While you can do this today, it i= s >>> not useful if you want to enforce storage QoS. >>> >>>> Are the SAN volumes by the management server or externally by the >> operator? >>> When the SolidFire plug-in is used, the SAN volumes are completely >> managed >>> by the management server (via the plug-in). There is no admin >> interaction. >>> This allows for a 1:1 mapping between a SAN volume and a CloudStack >> volume, >>> which is necessary for any storage vendor that supports true QoS. >>> >>>> >>>> I would like to clarify how these pieces are related and expected to >>>> operate. >>>> >>>> Thanks, >>>> -John >>>> >>>> On Jun 4, 2013, at 3:46 PM, Mike Tutkowski < >> mike.tutkowski@solidfire.com> >>>> wrote: >>>> >>>>> "In particular, how do we ensure that multiple VMs with provisioned >> IOPS >>>>> won't be cut off by the underlying storage." >>>>> >>>>> In the storage QoS world, we need to map a single SAN volume (LUN) to= a >>>>> single CloudStack volume. We cannot have multiple CloudStack volumes >>>>> sharing a single SAN volume and still guarantee QoS. >>>>> >>>>> If the user wants to have a single SAN volume house more than one >>>>> CloudStack volume, then can do that today without any of my plug-in >> code. >>>>> >>>>> >>>>> On Tue, Jun 4, 2013 at 1:43 PM, Mike Tutkowski < >>>> mike.tutkowski@solidfire.com >>>>>> wrote: >>>>> >>>>>> "The administrator will allocate a SAN volume for CloudStack's use >> onto >>>>>> which CloudStack volumes will be created." >>>>>> >>>>>> I think we crossed e-mails. :) >>>>>> >>>>>> Check out my recent e-mail on this. >>>>>> >>>>>> >>>>>> On Tue, Jun 4, 2013 at 1:41 PM, John Burwell >>>> wrote: >>>>>> >>>>>>> Mike, >>>>>>> >>>>>>> You are coming to the part which concerns me -- concepts from the >>>>>>> hypervisor are leaking into storage layer. >>>>>>> >>>>>>> Thanks, >>>>>>> -John >>>>>>> >>>>>>> On Jun 4, 2013, at 3:35 PM, Mike Tutkowski < >>>> mike.tutkowski@solidfire.com> >>>>>>> wrote: >>>>>>> >>>>>>>> The weird part is that the iSCSI type is today only used (as far a= s >> I >>>>>>> know) >>>>>>>> in regards to XenServer (when you have not PreSetup an SR). >>>>>>>> >>>>>>>> If you want to use your iSCSI volume from VMware, it uses the vmfs >>>> type. >>>>>>>> >>>>>>>> If you want to use your iSCSI volume from KVM, it uses the >>>>>>> SharedMountPoint >>>>>>>> type. >>>>>>>> >>>>>>>> So, I suppose mine and Edison's thinking here was to make a new ty= pe >>>> of >>>>>>>> storage to describe this dynamic ability Edison added into the >> storage >>>>>>>> framework. Maybe it should be more specificy, though: Dynamic_iSCS= I >>>>>>> versus, >>>>>>>> say, Dynamic_FC. >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Jun 4, 2013 at 1:27 PM, Mike Tutkowski < >>>>>>> mike.tutkowski@solidfire.com >>>>>>>>> wrote: >>>>>>>> >>>>>>>>> "The storage device itself shouldn't know or care that it is bein= g >>>> used >>>>>>>>> for a Xen SR -- simply be able to answer questions about it is >>>>>>> storing." >>>>>>>>> >>>>>>>>> I see...so your concern here is that the SolidFire plug-in needs = to >>>>>>> call >>>>>>>>> itself "Dynamic" storage so that the hypervisor logic knows to >> treat >>>>>>> it as >>>>>>>>> such. >>>>>>>>> >>>>>>>>> I'm totally open to removing that constraint and just calling it >>>> iSCSI >>>>>>> or >>>>>>>>> whatever. We would just need a way for the hypervisor attach logi= c >> to >>>>>>>>> detect this new requirement and perform the necessary activities. >>>>>>>>> >>>>>>>>> >>>>>>>>> On Tue, Jun 4, 2013 at 1:24 PM, John Burwell >>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Mike, >>>>>>>>>> >>>>>>>>>> See my responses in-line. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> -John >>>>>>>>>> >>>>>>>>>> On Jun 4, 2013, at 3:10 PM, Mike Tutkowski < >>>>>>> mike.tutkowski@solidfire.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> I'm trying to picture this: >>>>>>>>>>> >>>>>>>>>>> "Finally, while CloudStack may be able to manage a device, an >>>>>>> operator >>>>>>>>>> may >>>>>>>>>>> chose to leave it unmanaged by CloudStack (e.g. the device is >>>> shared >>>>>>> by >>>>>>>>>>> many services, and the operator has chosen to dedicate only a >>>> portion >>>>>>>>>> of it >>>>>>>>>>> to CloudStack). Does my reasoning make sense?" >>>>>>>>>>> >>>>>>>>>>> I guess I'm not sure how creating a SAN volume via the plug-in >>>>>>> (before >>>>>>>>>> an >>>>>>>>>>> attach request to the hypervisor) would work unless the >> hypervisor >>>>>>>>>> consumes >>>>>>>>>>> the SAN volume in the form of, say, an SR. >>>>>>>>>> >>>>>>>>>> My thinking is that, independent of CloudStack, an operator >>>> allocates >>>>>>> a >>>>>>>>>> chunk of a SAN to CloudStack, and exposes it through a LUN. Th= ey >>>>>>> simply >>>>>>>>>> want to turn control of that LUN over to CloudStack, but not all= ow >>>>>>>>>> CloudStack to allocate anymore LUNs. >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> As the attach logic stands prior to my changes, we would be >> passing >>>>>>> in a >>>>>>>>>>> SAN volume that does not have the necessary hypervisor support >>>> (like >>>>>>> an >>>>>>>>>> SR) >>>>>>>>>>> and the logic will fail. >>>>>>>>>>> >>>>>>>>>>> Are you thinking we should maybe have the storage framework >> itself >>>>>>>>>> detect >>>>>>>>>>> that such a SAN volume needs support from the hypervisor side a= nd >>>>>>> have >>>>>>>>>> it >>>>>>>>>>> call into the agent code specifically to create the SR before t= he >>>>>>> attach >>>>>>>>>>> logic runs in the agent code? >>>>>>>>>> >>>>>>>>>> I think the hypervisor management plugin should have a rich enou= gh >>>>>>>>>> interface to storage to determine available for volume storage. >> For >>>>>>> Xen, >>>>>>>>>> this interface would allow the interrogation of the device to >>>>>>> determine the >>>>>>>>>> SR is present. The storage device itself shouldn't know or car= e >>>>>>> that it >>>>>>>>>> is being used for a Xen SR -- simply be able to answer questions >>>>>>> about it >>>>>>>>>> is storing. >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Tue, Jun 4, 2013 at 1:01 PM, Mike Tutkowski < >>>>>>>>>> mike.tutkowski@solidfire.com >>>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> So, the flow is as follows: >>>>>>>>>>>> >>>>>>>>>>>> * The admin registers the SolidFire driver (which is a type of >>>>>>>>>> so-called >>>>>>>>>>>> Dynamic storage). Once this is done, a new Primary Storage sho= ws >>>> up >>>>>>> in >>>>>>>>>> the >>>>>>>>>>>> applicable zone. >>>>>>>>>>>> >>>>>>>>>>>> * The admin creates a Disk Offering that references the storag= e >>>> tag >>>>>>> of >>>>>>>>>> the >>>>>>>>>>>> newly created Primary Storage. >>>>>>>>>>>> >>>>>>>>>>>> * The end user creates a CloudStack volume. This leads to a ne= w >>>> row >>>>>>> in >>>>>>>>>> the >>>>>>>>>>>> cloud.volumes table. >>>>>>>>>>>> >>>>>>>>>>>> * The end user attaches the CloudStack volume to a VM (attach >>>> disk). >>>>>>>>>> This >>>>>>>>>>>> leads to the storage framework calling the plug-in to create a >> new >>>>>>>>>> volume >>>>>>>>>>>> on its storage system (in my case, a SAN). The plug-in also >>>> updates >>>>>>> the >>>>>>>>>>>> cloud.volumes row with applicable data (like the IQN of the SA= N >>>>>>>>>> volume). >>>>>>>>>>>> This plug-in code is only invoked if the CloudStack volume is = in >>>> the >>>>>>>>>>>> 'Allocated' state. After the attach, the volume will be in the >>>>>>> 'Ready' >>>>>>>>>>>> state (even after a detach disk) and the plug-in code will not >> be >>>>>>>>>> called >>>>>>>>>>>> again to create this SAN volume. >>>>>>>>>>>> >>>>>>>>>>>> * The hypervisor-attach logic is run and detects the CloudStac= k >>>>>>> volume >>>>>>>>>> to >>>>>>>>>>>> attach needs "assistance" in the form of a hypervisor data >>>> structure >>>>>>>>>> (ex. >>>>>>>>>>>> an SR on XenServer). >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:54 PM, Mike Tutkowski < >>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> "To ensure that we are in sync on terminology, volume, in the= se >>>>>>>>>>>>> definitions, refers to the physical allocation on the device, >>>>>>>>>> correct?" >>>>>>>>>>>>> >>>>>>>>>>>>> Yes...when I say 'volume', I try to mean 'SAN volume'. >>>>>>>>>>>>> >>>>>>>>>>>>> To refer to the 'volume' the end user can make in CloudStack,= I >>>>>>> try to >>>>>>>>>>>>> use 'CloudStack volume'. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:50 PM, Mike Tutkowski < >>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi John, >>>>>>>>>>>>>> >>>>>>>>>>>>>> What you say here may very well make sense, but I'm having a >>>> hard >>>>>>>>>> time >>>>>>>>>>>>>> envisioning it. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Perhaps we should draw Edison in on this conversation as he >> was >>>>>>> the >>>>>>>>>>>>>> initial person to suggest the approach I took. >>>>>>>>>>>>>> >>>>>>>>>>>>>> What do you think? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks! >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Tue, Jun 4, 2013 at 12:42 PM, John Burwell < >>>> jburwell@basho.com >>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Mike, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> It feels like we are combining two distinct concepts -- >> storage >>>>>>>>>> device >>>>>>>>>>>>>>> management and storage protocols. In both cases, we are >>>>>>>>>> communicating with >>>>>>>>>>>>>>> ISCSI, but one allows the system to create/delete volumes >>>>>>> (Dynamic) >>>>>>>>>> on the >>>>>>>>>>>>>>> device while the other requires the volume to be volume to = be >>>>>>>>>> managed >>>>>>>>>>>>>>> outside of the CloudStack context. To ensure that we are i= n >>>>>>> sync on >>>>>>>>>>>>>>> terminology, volume, in these definitions, refers to the >>>> physical >>>>>>>>>>>>>>> allocation on the device, correct? Minimally, we must be >> able >>>> to >>>>>>>>>>>>>>> communicate with a storage device to move bits from one pla= ce >>>> to >>>>>>>>>> another, >>>>>>>>>>>>>>> read bits, delete bits, etc. Optionally, a storage device >> may >>>> be >>>>>>>>>> able to >>>>>>>>>>>>>>> managed by CloudStack. Therefore, we can have a unmanaged >>>> iSCSI >>>>>>>>>> device >>>>>>>>>>>>>>> onto which we store a Xen SR, and we can have a managed >>>> SolidFire >>>>>>>>>> iSCSI >>>>>>>>>>>>>>> device on which CloudStack is capable of allocating LUNs an= d >>>>>>> storing >>>>>>>>>>>>>>> volumes. Finally, while CloudStack may be able to manage a >>>>>>> device, >>>>>>>>>> an >>>>>>>>>>>>>>> operator may chose to leave it unmanaged by CloudStack (e.g= . >>>> the >>>>>>>>>> device is >>>>>>>>>>>>>>> shared by many services, and the operator has chosen to >>>> dedicate >>>>>>>>>> only a >>>>>>>>>>>>>>> portion of it to CloudStack). Does my reasoning make sense= ? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Assuming my thoughts above are reasonable, it seems >> appropriate >>>>>>> to >>>>>>>>>>>>>>> strip the management concerns from StoragePoolType, add the >>>>>>> notion >>>>>>>>>> of a >>>>>>>>>>>>>>> storage device with an attached driver that indicates wheth= er >>>> or >>>>>>>>>> not is >>>>>>>>>>>>>>> managed by CloudStack, and establish a abstraction >>>> representing a >>>>>>>>>> physical >>>>>>>>>>>>>>> allocation on a device separate that is associated with a >>>> volume. >>>>>>>>>> With >>>>>>>>>>>>>>> these notions in place, hypervisor drivers can declare whic= h >>>>>>>>>> protocols they >>>>>>>>>>>>>>> support and when they encounter a device managed by >> CloudStack, >>>>>>>>>> utilize the >>>>>>>>>>>>>>> management operations exposed by the driver to automate >>>>>>> allocation. >>>>>>>>>> If >>>>>>>>>>>>>>> these thoughts/concepts make sense, then we can sit down an= d >>>>>>> drill >>>>>>>>>> down to >>>>>>>>>>>>>>> a more detailed design. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> -John >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Jun 3, 2013, at 5:25 PM, Mike Tutkowski < >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Here is the difference between the current iSCSI type and >> the >>>>>>>>>> Dynamic >>>>>>>>>>>>>>> type: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> iSCSI type: The admin has to go in and create a Primary >>>> Storage >>>>>>>>>> based >>>>>>>>>>>>>>> on >>>>>>>>>>>>>>>> the iSCSI type. At this point in time, the iSCSI volume mu= st >>>>>>> exist >>>>>>>>>> on >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>> storage system (it is pre-allocated). Future CloudStack >>>> volumes >>>>>>> are >>>>>>>>>>>>>>> created >>>>>>>>>>>>>>>> as VDIs on the SR that was created behind the scenes. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Dynamic type: The admin has to go in and create Primary >>>> Storage >>>>>>>>>> based >>>>>>>>>>>>>>> on a >>>>>>>>>>>>>>>> plug-in that will create and delete volumes on its storage >>>>>>> system >>>>>>>>>>>>>>>> dynamically (as is enabled via the storage framework). Whe= n >> a >>>>>>> user >>>>>>>>>>>>>>> wants to >>>>>>>>>>>>>>>> attach a CloudStack volume that was created, the framework >>>> tells >>>>>>>>>> the >>>>>>>>>>>>>>>> plug-in to create a new volume. After this is done, the >> attach >>>>>>>>>> logic >>>>>>>>>>>>>>> for >>>>>>>>>>>>>>>> the hypervisor in question is called. No hypervisor data >>>>>>> structure >>>>>>>>>>>>>>> exists >>>>>>>>>>>>>>>> at this point because the volume was just created. The >>>>>>> hypervisor >>>>>>>>>> data >>>>>>>>>>>>>>>> structure must be created. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:21 PM, Mike Tutkowski < >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com >>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> These are new terms, so I should probably have defined th= em >>>> up >>>>>>>>>> front >>>>>>>>>>>>>>> for >>>>>>>>>>>>>>>>> you. :) >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Static storage: Storage that is pre-allocated (ex. an adm= in >>>>>>>>>> creates a >>>>>>>>>>>>>>>>> volume on a SAN), then a hypervisor data structure is >> created >>>>>>> to >>>>>>>>>>>>>>> consume >>>>>>>>>>>>>>>>> the storage (ex. XenServer SR), then that hypervisor data >>>>>>>>>> structure >>>>>>>>>>>>>>> is >>>>>>>>>>>>>>>>> consumed by CloudStack. Disks (VDI) are later placed on >> this >>>>>>>>>>>>>>> hypervisor >>>>>>>>>>>>>>>>> data structure as needed. In these cases, the attach logi= c >>>>>>> assumes >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>> hypervisor data structure is already in place and simply >>>>>>> attaches >>>>>>>>>>>>>>> the VDI >>>>>>>>>>>>>>>>> on the hypervisor data structure to the VM in question. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Dynamic storage: Storage that is not pre-allocated. Inste= ad >>>> of >>>>>>>>>>>>>>>>> pre-existent storage, this could be a SAN (not a volume o= n >> a >>>>>>> SAN, >>>>>>>>>>>>>>> but the >>>>>>>>>>>>>>>>> SAN itself). The hypervisor data structure must be create= d >>>>>>> when an >>>>>>>>>>>>>>> attach >>>>>>>>>>>>>>>>> volume is performed because these types of volumes have n= ot >>>>>>> been >>>>>>>>>>>>>>> pre-hooked >>>>>>>>>>>>>>>>> up to such a hypervisor data structure by an admin. Once >> the >>>>>>>>>> attach >>>>>>>>>>>>>>> logic >>>>>>>>>>>>>>>>> creates, say, an SR on XenServer for this volume, it >> attaches >>>>>>> the >>>>>>>>>>>>>>> one and >>>>>>>>>>>>>>>>> only VDI within the SR to the VM in question. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:13 PM, John Burwell < >>>>>>> jburwell@basho.com> >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Mike, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> The current implementation of the Dynamic type attach >>>> behavior >>>>>>>>>>>>>>> works in >>>>>>>>>>>>>>>>>> terms of Xen ISCSI which why I ask about the difference. >>>>>>> Another >>>>>>>>>>>>>>> way to >>>>>>>>>>>>>>>>>> ask the question -- what is the definition of a Dynamic >>>>>>> storage >>>>>>>>>>>>>>> pool type? >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> -John >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 5:10 PM, Mike Tutkowski < >>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> >>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> As far as I know, the iSCSI type is uniquely used by >>>>>>> XenServer >>>>>>>>>>>>>>> when you >>>>>>>>>>>>>>>>>>> want to set up Primary Storage that is directly based o= n >> an >>>>>>>>>> iSCSI >>>>>>>>>>>>>>>>>> target. >>>>>>>>>>>>>>>>>>> This allows you to skip the step of going to the >> hypervisor >>>>>>> and >>>>>>>>>>>>>>>>>> creating a >>>>>>>>>>>>>>>>>>> storage repository based on that iSCSI target as >> CloudStack >>>>>>> does >>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>>> part >>>>>>>>>>>>>>>>>>> for you. I think this is only supported for XenServer. >> For >>>>>>> all >>>>>>>>>>>>>>> other >>>>>>>>>>>>>>>>>>> hypervisors, you must first go to the hypervisor and >>>> perform >>>>>>>>>> this >>>>>>>>>>>>>>> step >>>>>>>>>>>>>>>>>>> manually. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I don't really know what RBD is. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 2:13 PM, John Burwell < >>>>>>>>>> jburwell@basho.com> >>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Mike, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Reading through the code, what is the difference betwe= en >>>> the >>>>>>>>>>>>>>> ISCSI and >>>>>>>>>>>>>>>>>>>> Dynamic types? Why isn't RBD considered Dynamic? >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>> -John >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 3:46 PM, Mike Tutkowski < >>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> >>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> This new type of storage is defined in the >>>>>>>>>>>>>>> Storage.StoragePoolType >>>>>>>>>>>>>>>>>> class >>>>>>>>>>>>>>>>>>>>> (called Dynamic): >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> public static enum StoragePoolType { >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Filesystem(false), // local directory >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> NetworkFilesystem(true), // NFS or CIFS >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> IscsiLUN(true), // shared LUN, with a clusterfs >> overlay >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Iscsi(true), // for e.g., ZFS Comstar >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> ISO(false), // for iso image >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> LVM(false), // XenServer local LVM SR >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> CLVM(true), >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> RBD(true), >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> SharedMountPoint(true), >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> VMFS(true), // VMware VMFS storage >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> PreSetup(true), // for XenServer, Storage Pool is set >>>> up >>>>>>>>>> by >>>>>>>>>>>>>>>>>>>>> customers. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> EXT(false), // XenServer local EXT SR >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> OCFS2(true), >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Dynamic(true); // dynamic, zone-wide storage (ex. >>>>>>>>>> SolidFire) >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> boolean shared; >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> StoragePoolType(boolean shared) { >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> this.shared =3D shared; >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> public boolean isShared() { >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> return shared; >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:41 PM, Mike Tutkowski < >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com >>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> For example, let's say another storage company wants >> to >>>>>>>>>>>>>>> implement a >>>>>>>>>>>>>>>>>>>>>> plug-in to leverage its Quality of Service feature. = It >>>>>>> would >>>>>>>>>> be >>>>>>>>>>>>>>>>>> dynamic, >>>>>>>>>>>>>>>>>>>>>> zone-wide storage, as well. They would need only >>>>>>> implement a >>>>>>>>>>>>>>> storage >>>>>>>>>>>>>>>>>>>>>> plug-in as I've made the necessary changes to the >>>>>>>>>>>>>>> hypervisor-attach >>>>>>>>>>>>>>>>>>>> logic >>>>>>>>>>>>>>>>>>>>>> to support their plug-in. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:39 PM, Mike Tutkowski < >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote: >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Oh, sorry to imply the XenServer code is SolidFire >>>>>>> specific. >>>>>>>>>>>>>>> It is >>>>>>>>>>>>>>>>>> not. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> The XenServer attach logic is now aware of dynamic, >>>>>>>>>> zone-wide >>>>>>>>>>>>>>>>>> storage >>>>>>>>>>>>>>>>>>>>>>> (and SolidFire is an implementation of this kind of >>>>>>>>>> storage). >>>>>>>>>>>>>>> This >>>>>>>>>>>>>>>>>>>> kind of >>>>>>>>>>>>>>>>>>>>>>> storage is new to 4.2 with Edison's storage framewo= rk >>>>>>>>>> changes. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Edison created a new framework that supported the >>>>>>> creation >>>>>>>>>> and >>>>>>>>>>>>>>>>>> deletion >>>>>>>>>>>>>>>>>>>>>>> of volumes dynamically. However, when I visited wit= h >>>> him >>>>>>> in >>>>>>>>>>>>>>> Portland >>>>>>>>>>>>>>>>>>>> back >>>>>>>>>>>>>>>>>>>>>>> in April, we realized that it was not complete. We >>>>>>> realized >>>>>>>>>>>>>>> there >>>>>>>>>>>>>>>>>> was >>>>>>>>>>>>>>>>>>>>>>> nothing CloudStack could do with these volumes unle= ss >>>> the >>>>>>>>>>>>>>> attach >>>>>>>>>>>>>>>>>> logic >>>>>>>>>>>>>>>>>>>> was >>>>>>>>>>>>>>>>>>>>>>> changed to recognize this new type of storage and >>>> create >>>>>>> the >>>>>>>>>>>>>>>>>>>> appropriate >>>>>>>>>>>>>>>>>>>>>>> hypervisor data structure. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:28 PM, John Burwell < >>>>>>>>>>>>>>> jburwell@basho.com> >>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Mike, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> It is generally odd to me that any operation in th= e >>>>>>> Storage >>>>>>>>>>>>>>> layer >>>>>>>>>>>>>>>>>>>> would >>>>>>>>>>>>>>>>>>>>>>>> understand or care about details. I expect to see >> the >>>>>>>>>> Storage >>>>>>>>>>>>>>>>>>>> services >>>>>>>>>>>>>>>>>>>>>>>> expose a set of operations that can be >> composed/driven >>>>>>> by >>>>>>>>>> the >>>>>>>>>>>>>>>>>>>> Hypervisor >>>>>>>>>>>>>>>>>>>>>>>> implementations to allocate space/create structure= s >>>> per >>>>>>>>>> their >>>>>>>>>>>>>>>>>> needs. >>>>>>>>>>>>>>>>>>>> If >>>>>>>>>>>>>>>>>>>>>>>> we >>>>>>>>>>>>>>>>>>>>>>>> don't invert this dependency, we are going to end >>>> with a >>>>>>>>>>>>>>> massive >>>>>>>>>>>>>>>>>>>> n-to-n >>>>>>>>>>>>>>>>>>>>>>>> problem that will make the system increasingly >>>>>>> difficult to >>>>>>>>>>>>>>>>>> maintain >>>>>>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>>>>>> enhance. Am I understanding that the Xen specific >>>>>>>>>> SolidFire >>>>>>>>>>>>>>> code >>>>>>>>>>>>>>>>>> is >>>>>>>>>>>>>>>>>>>>>>>> located in the CitrixResourceBase class? >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>> -John >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 3:12 PM, Mike Tutkowski < >>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com >>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> To delve into this in a bit more detail: >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Prior to 4.2 and aside from one setup method for >>>>>>>>>> XenServer, >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>> admin >>>>>>>>>>>>>>>>>>>>>>>> had >>>>>>>>>>>>>>>>>>>>>>>>> to first create a volume on the storage system, >> then >>>> go >>>>>>>>>> into >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>> hypervisor >>>>>>>>>>>>>>>>>>>>>>>>> to set up a data structure to make use of the >> volume >>>>>>> (ex. >>>>>>>>>> a >>>>>>>>>>>>>>>>>> storage >>>>>>>>>>>>>>>>>>>>>>>>> repository on XenServer or a datastore on ESX(i))= . >>>> VMs >>>>>>> and >>>>>>>>>>>>>>> data >>>>>>>>>>>>>>>>>> disks >>>>>>>>>>>>>>>>>>>>>>>> then >>>>>>>>>>>>>>>>>>>>>>>>> shared this storage system's volume. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> With Edison's new storage framework, storage need >> no >>>>>>>>>> longer >>>>>>>>>>>>>>> be so >>>>>>>>>>>>>>>>>>>>>>>> static >>>>>>>>>>>>>>>>>>>>>>>>> and you can easily create a 1:1 relationship >> between >>>> a >>>>>>>>>>>>>>> storage >>>>>>>>>>>>>>>>>>>> system's >>>>>>>>>>>>>>>>>>>>>>>>> volume and the VM's data disk (necessary for >> storage >>>>>>>>>> Quality >>>>>>>>>>>>>>> of >>>>>>>>>>>>>>>>>>>>>>>> Service). >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> You can now write a plug-in that is called to >>>>>>> dynamically >>>>>>>>>>>>>>> create >>>>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>>>>>> delete >>>>>>>>>>>>>>>>>>>>>>>>> volumes as needed. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> The problem that the storage framework did not >>>> address >>>>>>> is >>>>>>>>>> in >>>>>>>>>>>>>>>>>> creating >>>>>>>>>>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>>>>>>> deleting the hypervisor-specific data structure >> when >>>>>>>>>>>>>>> performing an >>>>>>>>>>>>>>>>>>>>>>>>> attach/detach. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> That being the case, I've been enhancing it to do >> so. >>>>>>> I've >>>>>>>>>>>>>>> got >>>>>>>>>>>>>>>>>>>>>>>> XenServer >>>>>>>>>>>>>>>>>>>>>>>>> worked out and submitted. I've got ESX(i) in my >>>> sandbox >>>>>>>>>> and >>>>>>>>>>>>>>> can >>>>>>>>>>>>>>>>>>>> submit >>>>>>>>>>>>>>>>>>>>>>>> this >>>>>>>>>>>>>>>>>>>>>>>>> if we extend the 4.2 freeze date. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Does that help a bit? :) >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 1:03 PM, Mike Tutkowski < >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com >>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Hi John, >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> The storage plug-in - by itself - is hypervisor >>>>>>> agnostic. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> The issue is with the volume-attach logic (in th= e >>>>>>> agent >>>>>>>>>>>>>>> code). >>>>>>>>>>>>>>>>>> The >>>>>>>>>>>>>>>>>>>>>>>>> storage >>>>>>>>>>>>>>>>>>>>>>>>>> framework calls into the plug-in to have it >> create a >>>>>>>>>> volume >>>>>>>>>>>>>>> as >>>>>>>>>>>>>>>>>>>>>>>> needed, >>>>>>>>>>>>>>>>>>>>>>>>> but >>>>>>>>>>>>>>>>>>>>>>>>>> when the time comes to attach the volume to a >>>>>>> hypervisor, >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>> attach >>>>>>>>>>>>>>>>>>>>>>>>> logic >>>>>>>>>>>>>>>>>>>>>>>>>> has to be smart enough to recognize it's being >>>>>>> invoked on >>>>>>>>>>>>>>>>>> zone-wide >>>>>>>>>>>>>>>>>>>>>>>>> storage >>>>>>>>>>>>>>>>>>>>>>>>>> (where the volume has just been created) and >> create, >>>>>>>>>> say, a >>>>>>>>>>>>>>>>>> storage >>>>>>>>>>>>>>>>>>>>>>>>>> repository (for XenServer) or a datastore (for >>>>>>> VMware) to >>>>>>>>>>>>>>> make >>>>>>>>>>>>>>>>>> use >>>>>>>>>>>>>>>>>>>>>>>> of the >>>>>>>>>>>>>>>>>>>>>>>>>> volume that was just created. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> I've been spending most of my time recently maki= ng >>>> the >>>>>>>>>>>>>>> attach >>>>>>>>>>>>>>>>>> logic >>>>>>>>>>>>>>>>>>>>>>>> work >>>>>>>>>>>>>>>>>>>>>>>>>> in the agent code. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Does that clear it up? >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks! >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 12:48 PM, John Burwell < >>>>>>>>>>>>>>>>>> jburwell@basho.com> >>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Mike, >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why the the storage driver is >>>>>>> hypervisor >>>>>>>>>>>>>>>>>> specific? >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>> -John >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 1:21 PM, Mike Tutkowski < >>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, ultimately I would like to support all >>>>>>> hypervisors >>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>>>>>>>>>> CloudStack >>>>>>>>>>>>>>>>>>>>>>>>>>>> supports. I think I'm just out of time for 4.2 >> to >>>>>>> get >>>>>>>>>> KVM >>>>>>>>>>>>>>> in. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Right now this plug-in supports XenServer. >>>>>>> Depending on >>>>>>>>>>>>>>> what >>>>>>>>>>>>>>>>>> we do >>>>>>>>>>>>>>>>>>>>>>>>> with >>>>>>>>>>>>>>>>>>>>>>>>>>>> regards to 4.2 feature freeze, I have it worki= ng >>>> for >>>>>>>>>>>>>>> VMware in >>>>>>>>>>>>>>>>>> my >>>>>>>>>>>>>>>>>>>>>>>>>>> sandbox, >>>>>>>>>>>>>>>>>>>>>>>>>>>> as well. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, just to be clear, this is all in regards >> to >>>>>>> Disk >>>>>>>>>>>>>>>>>> Offerings. >>>>>>>>>>>>>>>>>>>>>>>> I >>>>>>>>>>>>>>>>>>>>>>>>>>> plan to >>>>>>>>>>>>>>>>>>>>>>>>>>>> support Compute Offerings post 4.2. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamiso= n >>>>>>> Damage >>>>>>>>>> < >>>>>>>>>>>>>>>>>>>>>>>>> kelcey@bbits.ca >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there any plan on supporting KVM in the >> patch >>>>>>> cycle >>>>>>>>>>>>>>> post >>>>>>>>>>>>>>>>>> 4.2? >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ----- Original Message ----- >>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: "Mike Tutkowski" < >>>>>>> mike.tutkowski@solidfire.com> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent: Monday, June 3, 2013 10:12:32 AM >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to >> MASTER >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree on merging Wei's feature first, then >>>> mine. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> If his feature is for KVM only, then it is a >> non >>>>>>> issue >>>>>>>>>>>>>>> as I >>>>>>>>>>>>>>>>>> don't >>>>>>>>>>>>>>>>>>>>>>>>>>> support >>>>>>>>>>>>>>>>>>>>>>>>>>>>> KVM in 4.2. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU < >>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com> >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> John, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the billing, as no one works on billing >> now, >>>>>>>>>> users >>>>>>>>>>>>>>> need >>>>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>>>>>>>>>> calculate >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the billing by themselves. They can get the >>>>>>>>>>>>>>> service_offering >>>>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_offering of a VMs and volumes for >>>>>>> calculation. >>>>>>>>>> Of >>>>>>>>>>>>>>> course >>>>>>>>>>>>>>>>>>>>>>>> it is >>>>>>>>>>>>>>>>>>>>>>>>>>>>> better >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to tell user the exact limitation value of >>>>>>> individual >>>>>>>>>>>>>>> volume, >>>>>>>>>>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>>>>>>>>> network >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rate limitation for nics as well. I can work >> on >>>> it >>>>>>>>>>>>>>> later. Do >>>>>>>>>>>>>>>>>> you >>>>>>>>>>>>>>>>>>>>>>>>>>> think it >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is a part of I/O throttling? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry my misunstand the second the question. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Agree with what you said about the two >> features. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU < >>>>>>>>>>>>>>> ustcweizhou@gmail.com >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John, Mike >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope Mike's aswer helps you. I am trying >> to >>>>>>>>>> adding >>>>>>>>>>>>>>> more. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) I think billing should depend on IO >>>>>>> statistics >>>>>>>>>>>>>>> rather >>>>>>>>>>>>>>>>>> than >>>>>>>>>>>>>>>>>>>>>>>>> IOPS >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation. Please review disk_io_stat if >> you >>>>>>> have >>>>>>>>>>>>>>> time. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_stat >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the IO statistics including bytes/iops >>>>>>>>>> read/write >>>>>>>>>>>>>>> for >>>>>>>>>>>>>>>>>> an >>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> virtual machine. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Going by the AWS model, customers are bille= d >>>> more >>>>>>>>>> for >>>>>>>>>>>>>>>>>> volumes >>>>>>>>>>>>>>>>>>>>>>>> with >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioned IOPS, as well as, for those >>>>>>> operations ( >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://aws.amazon.com/ebs/). I would >> imagine >>>>>>> our >>>>>>>>>>>>>>> users >>>>>>>>>>>>>>>>>> would >>>>>>>>>>>>>>>>>>>>>>>> like >>>>>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> option to employ similar cost models. Coul= d >> an >>>>>>>>>>>>>>> operator >>>>>>>>>>>>>>>>>>>>>>>> implement >>>>>>>>>>>>>>>>>>>>>>>>>>>>> such a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> billing model in the current patch? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) Do you mean IOPS runtime change? KVM >>>>>>> supports >>>>>>>>>>>>>>> setting >>>>>>>>>>>>>>>>>>>>>>>> IOPS/BPS >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> limitation for a running virtual machine >>>> through >>>>>>>>>>>>>>> command >>>>>>>>>>>>>>>>>> line. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> However, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CloudStack does not support changing the >>>>>>> parameters >>>>>>>>>>>>>>> of a >>>>>>>>>>>>>>>>>>>>>>>> created >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> offering >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (computer offering or disk offering). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I meant at the Java interface level. I >>>> apologize >>>>>>>>>> for >>>>>>>>>>>>>>> being >>>>>>>>>>>>>>>>>>>>>>>>> unclear. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we more generalize allocation algorithms >> with a >>>>>>> set >>>>>>>>>> of >>>>>>>>>>>>>>>>>>>>>>>> interfaces >>>>>>>>>>>>>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> describe the service guarantees provided by= a >>>>>>>>>> resource? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) It is a good question. Maybe it is >> better >>>> to >>>>>>>>>>>>>>> commit >>>>>>>>>>>>>>>>>> Mike's >>>>>>>>>>>>>>>>>>>>>>>>> patch >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling as Mike needs to consid= er >>>> the >>>>>>>>>>>>>>>>>> limitation in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> type, I think. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will expand on my thoughts in a later >>>> response >>>>>>> to >>>>>>>>>>>>>>> Mike >>>>>>>>>>>>>>>>>>>>>>>> regarding >>>>>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> touch points between these two features. I >>>> think >>>>>>>>>> that >>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will need to be merged before SolidFire, bu= t >> I >>>>>>> think >>>>>>>>>>>>>>> we need >>>>>>>>>>>>>>>>>>>>>>>> closer >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> coordination between the branches (possibly >>>> have >>>>>>>>>> have >>>>>>>>>>>>>>>>>> solidfire >>>>>>>>>>>>>>>>>>>>>>>>> track >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling) to coordinate on this >>>> issue. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Wei >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/3 John Burwell >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The things I want to understand are the >>>>>>> following: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) Is there value in capturing IOPS >> policies >>>> be >>>>>>>>>>>>>>> captured >>>>>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>>>>>>>>> a >>>>>>>>>>>>>>>>>>>>>>>>>>>>> common >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data model (e.g. for billing/usage >> purposes, >>>>>>>>>>>>>>> expressing >>>>>>>>>>>>>>>>>>>>>>>>> offerings). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Should there be a common interface mod= el >>>> for >>>>>>>>>>>>>>> reasoning >>>>>>>>>>>>>>>>>>>>>>>> about >>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOP >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provisioning at runtime? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) How are conflicting provisioned IOPS >>>>>>>>>>>>>>> configurations >>>>>>>>>>>>>>>>>>>>>>>> between >>>>>>>>>>>>>>>>>>>>>>>>> a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hypervisor and storage device reconciled? >> In >>>>>>>>>>>>>>> particular, >>>>>>>>>>>>>>>>>> a >>>>>>>>>>>>>>>>>>>>>>>>>>> scenario >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> where >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is lead to believe (and billed) for more >> IOPS >>>>>>>>>>>>>>> configured >>>>>>>>>>>>>>>>>> for >>>>>>>>>>>>>>>>>>>>>>>> a VM >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage device has been configured to >>>> deliver. >>>>>>>>>>>>>>> Another >>>>>>>>>>>>>>>>>>>>>>>> scenario >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consistent configuration between a VM and= a >>>>>>>>>> storage >>>>>>>>>>>>>>>>>> device at >>>>>>>>>>>>>>>>>>>>>>>>>>>>> creation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time, but a later modification to storage >>>>>>> device >>>>>>>>>>>>>>>>>> introduces >>>>>>>>>>>>>>>>>>>>>>>>> logical >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> inconsistency. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowsk= i >> < >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi John, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I believe Wei's feature deals with >>>> controlling >>>>>>> the >>>>>>>>>>>>>>> max >>>>>>>>>>>>>>>>>>>>>>>> number of >>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hypervisor side. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My feature is focused on controlling IOPS >>>> from >>>>>>> the >>>>>>>>>>>>>>> storage >>>>>>>>>>>>>>>>>>>>>>>> system >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> side. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I hope that helps. :) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John >> Burwell >>>> < >>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My opinion is that no features should be >>>>>>> merged >>>>>>>>>>>>>>> until all >>>>>>>>>>>>>>>>>>>>>>>>>>>>> functional >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues have been resolved and it is read= y >> to >>>>>>> turn >>>>>>>>>>>>>>> over to >>>>>>>>>>>>>>>>>>>>>>>> test. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total Ops vs discrete read/write ops iss= ue >>>> is >>>>>>>>>>>>>>> addressed >>>>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> re-reviewed >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, I don't think this criteria has be= en >>>>>>>>>>>>>>> satisfied. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, how does this work >>>> intersect/compliment >>>>>>> the >>>>>>>>>>>>>>>>>> SolidFire >>>>>>>>>>>>>>>>>>>>>>>>> patch >>>>>>>>>>>>>>>>>>>>>>>>>>> ( >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/)? >> As I >>>>>>>>>>>>>>> understand >>>>>>>>>>>>>>>>>> it >>>>>>>>>>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>>>>>>>>>>>>>> work >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also involves provisioned IOPS. I would >>>> like >>>>>>> to >>>>>>>>>>>>>>> ensure >>>>>>>>>>>>>>>>>> we >>>>>>>>>>>>>>>>>>>>>>>> don't >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scenario where provisioned IOPS in KVM a= nd >>>>>>>>>>>>>>> SolidFire are >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unnecessarily >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> incompatible. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU < >>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure. I will change it next week. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/1 Wido den Hollander < >> wido@widodh.nl >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Exactly. I have pushed the features into >>>>>>> master. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If anyone object thems for technical >> reason >>>>>>> till >>>>>>>>>>>>>>> Monday, >>>>>>>>>>>>>>>>>> I >>>>>>>>>>>>>>>>>>>>>>>> will >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> revert >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the sake of clarity I just want to >>>> mention >>>>>>>>>> again >>>>>>>>>>>>>>>>>> that we >>>>>>>>>>>>>>>>>>>>>>>>>>> should >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the total IOps to R/W IOps asap so that = we >>>>>>> never >>>>>>>>>>>>>>> release >>>>>>>>>>>>>>>>>> a >>>>>>>>>>>>>>>>>>>>>>>>> version >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> only total IOps. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You laid the groundwork for the I/O >>>> throttling >>>>>>>>>> and >>>>>>>>>>>>>>> that's >>>>>>>>>>>>>>>>>>>>>>>> great! >>>>>>>>>>>>>>>>>>>>>>>>>>> We >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> however prevent that we create legacy fr= om >>>> day >>>>>>>>>> #1. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander < >>>> wido@widodh.nl> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:59 PM, John Burwell >> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 -- this enhancement must to discretel= y >>>>>>> support >>>>>>>>>>>>>>> read >>>>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>>>>>> write >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOPS. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't see how it could be fixed later >>>> because >>>>>>> I >>>>>>>>>>>>>>> don't see >>>>>>>>>>>>>>>>>>>>>>>> how we >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correctly >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> split total IOPS into read and write. >>>>>>>>>> Therefore, we >>>>>>>>>>>>>>>>>> would >>>>>>>>>>>>>>>>>>>>>>>> be >>>>>>>>>>>>>>>>>>>>>>>>>>> stuck >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total unless/until we decided to break >>>>>>> backwards >>>>>>>>>>>>>>>>>>>>>>>> compatibility. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What Wei meant was merging it into maste= r >>>> now >>>>>>> so >>>>>>>>>>>>>>> that it >>>>>>>>>>>>>>>>>>>>>>>> will go >>>>>>>>>>>>>>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2 branch and add Read / Write IOps >> before >>>>>>> the >>>>>>>>>> 4.2 >>>>>>>>>>>>>>>>>> release >>>>>>>>>>>>>>>>>>>>>>>> so >>>>>>>>>>>>>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4.2 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> will be released with Read and Write >> instead >>>>>>> of >>>>>>>>>>>>>>> Total >>>>>>>>>>>>>>>>>> IOps. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is to make the May 31st feature >> freeze >>>>>>> date. >>>>>>>>>>>>>>> But if >>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>> window >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> moves >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (see other threads) then it won't be >>>> necessary >>>>>>>>>> to do >>>>>>>>>>>>>>>>>> that. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also completely agree that there is no >>>>>>>>>> association >>>>>>>>>>>>>>>>>> between >>>>>>>>>>>>>>>>>>>>>>>>>>>>> network >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> disk I/O. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On May 31, 2013, at 9:51 AM, Wido den >>>>>>> Hollander < >>>>>>>>>>>>>>>>>>>>>>>> wido@widodh.nl >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wido, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks. Good question. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I thought about at the beginning. >> Finally I >>>>>>>>>>>>>>> decided to >>>>>>>>>>>>>>>>>>>>>>>> ignore >>>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> difference of read and write mainly >> because >>>>>>> the >>>>>>>>>>>>>>> network >>>>>>>>>>>>>>>>>>>>>>>>> throttling >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> care the difference of sent and received >>>>>>> bytes as >>>>>>>>>>>>>>> well. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That reasoning seems odd. Networking and >>>> disk >>>>>>> I/O >>>>>>>>>>>>>>>>>> completely >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> different. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Disk I/O is much more expensive in most >>>>>>>>>> situations >>>>>>>>>>>>>>> then >>>>>>>>>>>>>>>>>>>>>>>> network >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bandwith. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Implementing it will be some copy-paste >>>> work. >>>>>>> It >>>>>>>>>>>>>>> could be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implemented in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> few days. For the deadline of feature >>>> freeze, >>>>>>> I >>>>>>>>>> will >>>>>>>>>>>>>>>>>>>>>>>> implement >>>>>>>>>>>>>>>>>>>>>>>>> it >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that , if needed. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It think it's a feature we can't miss. B= ut >>>> if >>>>>>> it >>>>>>>>>>>>>>> goes >>>>>>>>>>>>>>>>>> into >>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>> 4.2 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> window we have to make sure we don't >> release >>>>>>> with >>>>>>>>>>>>>>> only >>>>>>>>>>>>>>>>>> total >>>>>>>>>>>>>>>>>>>>>>>>> IOps >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fix >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it in 4.3, that will confuse users. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/5/31 Wido den Hollander < >>>> wido@widodh.nl> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Wei, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to merge disk_io_throttling >>>>>>> branch >>>>>>>>>> into >>>>>>>>>>>>>>>>>> master. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If nobody object, I will merge into mast= er >>>> in >>>>>>> 48 >>>>>>>>>>>>>>> hours. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The purpose is : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual machines are running on the same >>>>>>> storage >>>>>>>>>>>>>>> device >>>>>>>>>>>>>>>>>>>>>>>> (local >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage or >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share strage). Because of the rate >>>> limitation >>>>>>> of >>>>>>>>>>>>>>> device >>>>>>>>>>>>>>>>>>>>>>>> (such as >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> iops), if >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> one VM has large disk operation, it may >>>> affect >>>>>>>>>> the >>>>>>>>>>>>>>> disk >>>>>>>>>>>>>>>>>>>>>>>>>>> performance >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other VMs running on the same storage >>>> device. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is neccesary to set the maximum rate >> and >>>>>>> limit >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>> disk >>>>>>>>>>>>>>>>>>>>>>>> I/O >>>>>>>>>>>>>>>>>>>>>>>>> of >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VMs. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Looking at the code I see you make no >>>>>>> difference >>>>>>>>>>>>>>> between >>>>>>>>>>>>>>>>>>>>>>>> Read >>>>>>>>>>>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Qemu and libvirt support setting both a >>>>>>> different >>>>>>>>>>>>>>> rate >>>>>>>>>>>>>>>>>> for >>>>>>>>>>>>>>>>>>>>>>>> Read >>>>>>>>>>>>>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps which could benefit a lot of users. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's also strange, in the polling side y= ou >>>>>>>>>> collect >>>>>>>>>>>>>>> both >>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>> Read >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps, but on the throttling side you onl= y >> go >>>>>>> for >>>>>>>>>> a >>>>>>>>>>>>>>> global >>>>>>>>>>>>>>>>>>>>>>>> value. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Write IOps are usually much more expensi= ve >>>>>>> then >>>>>>>>>> Read >>>>>>>>>>>>>>>>>> IOps, >>>>>>>>>>>>>>>>>>>>>>>> so it >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like a valid use-case where that an admi= n >>>>>>> would >>>>>>>>>> set >>>>>>>>>>>>>>> a >>>>>>>>>>>>>>>>>> lower >>>>>>>>>>>>>>>>>>>>>>>>> value >>>>>>>>>>>>>>>>>>>>>>>>>>>>> for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IOps vs Read IOps. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Since this only supports KVM at this >> point I >>>>>>>>>> think >>>>>>>>>>>>>>> it >>>>>>>>>>>>>>>>>> would >>>>>>>>>>>>>>>>>>>>>>>> be >>>>>>>>>>>>>>>>>>>>>>>>> of >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> great >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> value to at least have the mechanism in >>>> place >>>>>>> to >>>>>>>>>>>>>>> support >>>>>>>>>>>>>>>>>>>>>>>> both, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementing >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this later would be a lot of work. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If a hypervisor doesn't support setting >>>>>>> different >>>>>>>>>>>>>>> values >>>>>>>>>>>>>>>>>> for >>>>>>>>>>>>>>>>>>>>>>>>> read >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> write you can always sum both up and set >>>> that >>>>>>> as >>>>>>>>>> the >>>>>>>>>>>>>>>>>> total >>>>>>>>>>>>>>>>>>>>>>>>> limit. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you explain why you implemented it >> this >>>>>>> way? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wido >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The feature includes: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the maximum rate of VMs (in >>>>>>>>>> disk_offering, >>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>>>>>> global >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configuration) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) change the maximum rate of VMs >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) limit the disk rate (total bps and >> iops) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRA ticket: >> https://issues.apache.org/**** >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192>>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/**** >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jira/browse/CLOUDSTACK-1192< >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1192< >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <** >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192< >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FS (I will update later) : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>> >> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/****** >>>>>>>>>>>>>>> < >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> < >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/** >>>>>>>>>>>>>>>>>>>>>>>>> < >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/** >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling>>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/**** >>>>>>>>>>>>>>>>>>>>>>>>>>>>> < >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling< >>>>>>>>>>>>>>>>>> https://cwiki. >>>>>>>>>>>>>>>>>>>>>>>> ** >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>> >> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling >>>>>>>>>>>>>>>>>>>>>>>>>>>>> < >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>> >>>>>>> >>>> >> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Thrott= ling >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Merge check list :- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Did you check the branch's RAT executi= on >>>>>>>>>> success? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Are there new dependencies introduced? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What automated testing (unit and >>>>>>> integration) >>>>>>>>>> is >>>>>>>>>>>>>>>>>> included >>>>>>>>>>>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feature? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Unit tests are added. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * What testing has been done to check fo= r >>>>>>>>>> potential >>>>>>>>>>>>>>>>>>>>>>>> regressions? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (1) set the bytes rate and IOPS rate on >>>>>>>>>> CloudStack >>>>>>>>>>>>>>> UI. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (2) VM operations, including >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deploy, stop, start, reboot, destroy, >>>> expunge. >>>>>>>>>>>>>>> migrate, >>>>>>>>>>>>>>>>>>>>>>>> restore >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (3) Volume operations, including >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Attach, Detach >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To review the code, you can try >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> git diff >>>>>>>>>>>>>>> c30057635d04a2396f84c588127d7e******be42e503a7 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>> f2e5591b710d04cc86815044f5823e******73a4a58944 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wei >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>> >> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/****** >>>>>>>>>>>>>>> < >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> < >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/** >>>>>>>>>>>>>>>>>>>>>>>>> < >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/** >>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> VM+Disk+IO+Throttling>>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org/confluence/**** >>>>>>>>>>>>>>>>>>>>>>>>>>>>> < >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/**> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling< >>>>>>>>>>>>>>>>>> https://cwiki. >>>>>>>>>>>>>>>>>>>>>>>> ** >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>> >> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling >>>>>>>>>>>>>>>>>>>>>>>>>>>>> < >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>> >>>>>>> >>>> >> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Thrott= ling >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301 >>>>>>>>>>>>>>>>>>>>>>>> < >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301 >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****jira/browse/CLOUDSTACK-1301< >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/**jira/**browse/CLOUDSTACK-1301< >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <** >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301< >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> issues.apache.org/****jira/**browse/CLOUDSTACK-207= 1 >> < >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071 >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> **< >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071 >>>>>>>>>>>>>>>>>> < >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <** >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-207= 1 >> < >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> issues.apache.org/jira/**browse/CLOUDSTACK-2071< >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (**CLOUDSTACK-1301 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - VM Disk I/O Throttling) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski* >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire >> Inc.* >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud< >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=3Dplay> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *=99* >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski* >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302 >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the >>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud< >>>>>>>>>> http://solidfire.com/solution/overview/?video=3Dplay >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *=99* >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski* >>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com >>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302 >>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the >>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud< >>>>>>>>>> http://solidfire.com/solution/overview/?video=3Dplay> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *=99* >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski* >>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com >>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302 >>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud< >>>>>>>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=3Dp= lay >>> >>>>>>>>>>>>>>>>>>>>>>>>>> *=99* >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski* >>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com >>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302 >>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the >>>>>>>>>>>>>>>>>>>>>>>>> cloud< >>>>>>> http://solidfire.com/solution/overview/?video=3Dplay> >>>>>>>>>>>>>>>>>>>>>>>>> *=99* >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski* >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302 >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud< >>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=3Dplay> >>>>>>>>>>>>>>>>>>>>>>> *=99* >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski* >>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com >>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302 >>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud< >>>>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=3Dplay> >>>>>>>>>>>>>>>>>>>>>> *=99* >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski* >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302 >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the >>>>>>>>>>>>>>>>>>>>> cloud< >> http://solidfire.com/solution/overview/?video=3Dplay >>>>> >>>>>>>>>>>>>>>>>>>>> *=99* >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>> *Mike Tutkowski* >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com >>>>>>>>>>>>>>>>>>> o: 303.746.7302 >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the >>>>>>>>>>>>>>>>>>> cloud>> >>>>>>>>>>>>>>>>>>> *=99* >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>> *Mike Tutkowski* >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com >>>>>>>>>>>>>>>>> o: 303.746.7302 >>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud< >>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=3Dplay> >>>>>>>>>>>>>>>>> *=99* >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>> *Mike Tutkowski* >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com >>>>>>>>>>>>>>>> o: 303.746.7302 >>>>>>>>>>>>>>>> Advancing the way the world uses the >>>>>>>>>>>>>>>> cloud >>>>>>>>>>>>>>>> *=99* >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> *Mike Tutkowski* >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com >>>>>>>>>>>>>> o: 303.746.7302 >>>>>>>>>>>>>> Advancing the way the world uses the cloud< >>>>>>>>>> http://solidfire.com/solution/overview/?video=3Dplay> >>>>>>>>>>>>>> *=99* >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> *Mike Tutkowski* >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com >>>>>>>>>>>>> o: 303.746.7302 >>>>>>>>>>>>> Advancing the way the world uses the cloud< >>>>>>>>>> http://solidfire.com/solution/overview/?video=3Dplay> >>>>>>>>>>>>> *=99* >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> *Mike Tutkowski* >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>>>>>>>> e: mike.tutkowski@solidfire.com >>>>>>>>>>>> o: 303.746.7302 >>>>>>>>>>>> Advancing the way the world uses the cloud< >>>>>>>>>> http://solidfire.com/solution/overview/?video=3Dplay> >>>>>>>>>>>> *=99* >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> *Mike Tutkowski* >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>>>>>>> e: mike.tutkowski@solidfire.com >>>>>>>>>>> o: 303.746.7302 >>>>>>>>>>> Advancing the way the world uses the >>>>>>>>>>> cloud >>>>>>>>>>> *=99* >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> *Mike Tutkowski* >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>>>>> e: mike.tutkowski@solidfire.com >>>>>>>>> o: 303.746.7302 >>>>>>>>> Advancing the way the world uses the cloud< >>>>>>> http://solidfire.com/solution/overview/?video=3Dplay> >>>>>>>>> *=99* >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> *Mike Tutkowski* >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>>>> e: mike.tutkowski@solidfire.com >>>>>>>> o: 303.746.7302 >>>>>>>> Advancing the way the world uses the >>>>>>>> cloud >>>>>>>> *=99* >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Mike Tutkowski* >>>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>> e: mike.tutkowski@solidfire.com >>>>>> o: 303.746.7302 >>>>>> Advancing the way the world uses the cloud< >>>> http://solidfire.com/solution/overview/?video=3Dplay> >>>>>> *=99* >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> *Mike Tutkowski* >>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>> e: mike.tutkowski@solidfire.com >>>>> o: 303.746.7302 >>>>> Advancing the way the world uses the >>>>> cloud >>>>> *=99* >>>> >>>> >>> >>> >>> -- >>> *Mike Tutkowski* >>> *Senior CloudStack Developer, SolidFire Inc.* >>> e: mike.tutkowski@solidfire.com >>> o: 303.746.7302 >>> Advancing the way the world uses the >>> cloud >>> *=99* >> >> > > > -- > *Mike Tutkowski* > *Senior CloudStack Developer, SolidFire Inc.* > e: mike.tutkowski@solidfire.com > o: 303.746.7302 > Advancing the way the world uses the > cloud > *=99*