Return-Path: X-Original-To: apmail-cloudstack-dev-archive@www.apache.org Delivered-To: apmail-cloudstack-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id DBF32108AC for ; Wed, 12 Jun 2013 17:02:27 +0000 (UTC) Received: (qmail 77866 invoked by uid 500); 12 Jun 2013 17:02:27 -0000 Delivered-To: apmail-cloudstack-dev-archive@cloudstack.apache.org Received: (qmail 77833 invoked by uid 500); 12 Jun 2013 17:02:27 -0000 Mailing-List: contact dev-help@cloudstack.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cloudstack.apache.org Delivered-To: mailing list dev@cloudstack.apache.org Received: (qmail 77825 invoked by uid 99); 12 Jun 2013 17:02:27 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 12 Jun 2013 17:02:27 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of mike.tutkowski@solidfire.com designates 209.85.214.179 as permitted sender) Received: from [209.85.214.179] (HELO mail-ob0-f179.google.com) (209.85.214.179) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 12 Jun 2013 17:02:23 +0000 Received: by mail-ob0-f179.google.com with SMTP id xk17so13531272obc.38 for ; Wed, 12 Jun 2013 10:02:02 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:x-gm-message-state; bh=7gWRgMsE8293ph//ZH/z9eqhkv18lpT5+X6ls7+kj5w=; b=mO9vcRtkT4V5BD1nfq+MYWkU3vCD4nK3NhipBYo9xMVKR8UNwAYBrbwGRwu3XpQcUp TYnC3HNHtYi89LvbAHQx5V4VN2w4X1dM+CVRioKeE52PTdEyusiYmLQPrHHlleptjEV8 B5eVlmUiFFlwSlKB98/hW/w6PvuuTEybgyjIg9I2p268MZhhbp/gqZyw94hmdGI0IJE+ jyvUVkfk9r/4AYP8t7K3BWkgqd9+QZOxL8/j8Dyo5UcsULBzlVsD3ASIDct6DsYG0DF6 Jd/7or93XprdvW9dFxQEMj4QSjB1KCfsZvb2YefKWkWXE/l1HG8ifZCNFuaQiE5h5Dxi /6OQ== MIME-Version: 1.0 X-Received: by 10.60.62.162 with SMTP id z2mr15743656oer.140.1371056522554; Wed, 12 Jun 2013 10:02:02 -0700 (PDT) Received: by 10.182.10.66 with HTTP; Wed, 12 Jun 2013 10:02:02 -0700 (PDT) In-Reply-To: <1544949155485503655@unknownmsgid> References: <12535A06-F161-4739-855F-99B4A001EFAF@basho.com> <77B337AF224FD84CBF8401947098DD87039664@SJCPEX01CL01.citrite.net> <77B337AF224FD84CBF8401947098DD870396BD@SJCPEX01CL01.citrite.net> <77B337AF224FD84CBF8401947098DD8703977F@SJCPEX01CL01.citrite.net> <77B337AF224FD84CBF8401947098DD8704A7E8@SJCPEX01CL01.citrite.net> <4BC4E13F-B99C-44ED-B6FA-81CB9373A80F@basho.com> <9C87E699-FE4B-44D8-986C-FE43DED33E2D@basho.com> <77B337AF224FD84CBF8401947098DD8704BA0E@SJCPEX01CL01.citrite.net> <24155E6C-01E9-4ECC-93E9-EA552B8F0D87@basho.com> <77B337AF224FD84CBF8401947098DD8704BAB8@SJCPEX01CL01.citrite.net> <1130E63E-C93A-45C4-A96D-F375E9BB1C4F@basho.com> <77B337AF224FD84CBF8401947098DD8704BB28@SJCPEX01CL01.citrite.net> <018F234D-0552-4B0D-BF73-AB25876D960B@basho.com> <1150084447960098865@unknownmsgid> <1544949155485503655@unknownmsgid> Date: Wed, 12 Jun 2013 11:02:02 -0600 Message-ID: Subject: Re: [MERGE] disk_io_throttling to MASTER From: Mike Tutkowski To: "dev@cloudstack.apache.org" Content-Type: multipart/alternative; boundary=047d7b673d1afe82bf04def7f8c7 X-Gm-Message-State: ALoCoQkoxJcpEWQdCbe18Wchu0h4iU7BC4fnFIMUCnVjN9wYiYvMLt1zxEEQJx5xPQ2wxRSZmfit X-Virus-Checked: Checked by ClamAV on apache.org --047d7b673d1afe82bf04def7f8c7 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi John, Wei and I have discussed making the two features mutually exclusive. We agree with you that only one should be active at a time. We plan to implement in the GUI a mechanism (maybe radio buttons) to turn his feature on and mine off and vice versa. I was thinking if I wait until he checks in his code, then I update and merge that I will be the person resolving merge conflicts in the JavaScript code (there shouldn't be a problem in the Java code) as opposed to putting that work on someone else. Let me know what you think. Oh, was going to ask you what "FS" stands for here. Thanks! On Wed, Jun 12, 2013 at 10:56 AM, John Burwell wrote: > Mike, > > How have Wei and you resolved the issue of conflicting QoS mechanisms > between the Hypervisor and Storage layers? Have the affected FSs been > updated with that decision? > > In terms of merge timing, can you describe the dependencies between the > patches? > > Thanks, > -John > > > > > On Jun 12, 2013, at 12:43 PM, Mike Tutkowski > wrote: > > > No problem, John. > > > > I still want to re-review it by myself before coming up with a new patc= h > > file. > > > > Also, maybe I should first wait for Wei's changes to be checked in and > > merge those into mine before generating a new patch file? > > > > > > On Wed, Jun 12, 2013 at 10:40 AM, John Burwell > wrote: > > > >> Mike, > >> > >> I just realized that I forgot to publish my review. I am offline ATM, > >> but I will publish it in the next couple of hours. > >> > >> Do you plan to update your the patch in Review Board? > >> > >> Sorry for the oversight, > >> -John > >> > >> > >> > >> > >> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski > >> wrote: > >> > >>> Hi Edison, John, and Wei (and whoever else is reading this :) ), > >>> > >>> Just an FYI that I believe I have implemented all the areas we wanted > >>> addressed. > >>> > >>> I plan to review the code again tomorrow morning or afternoon, then > send > >> in > >>> another patch. > >>> > >>> Thanks for all the work on this everyone! > >>> > >>> > >>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski < > >>> mike.tutkowski@solidfire.com> wrote: > >>> > >>>> Sure, that sounds good. > >>>> > >>>> > >>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU > >> wrote: > >>>> > >>>>> Hi Mike, > >>>>> > >>>>> It looks the two feature do not have many conflicts in Java code, > >> except > >>>>> the cloudstack UI. > >>>>> If you do not mind, I will merge disk_io_throttling branch into > master > >>>>> this > >>>>> week, so that you can develop based on it. > >>>>> > >>>>> -Wei > >>>>> > >>>>> > >>>>> 2013/6/11 Mike Tutkowski > >>>>> > >>>>>> Hey John, > >>>>>> > >>>>>> The SolidFire patch does not depend on the object_store branch, bu= t > - > >> as > >>>>>> Edison mentioned - it might be easier if we merge the SolidFire > branch > >>>>> into > >>>>>> the object_store branch before object_store goes into master. > >>>>>> > >>>>>> I'm not sure how the disk_io_throttling fits into this merge > strategy. > >>>>>> Perhaps Wei can chime in on that. > >>>>>> > >>>>>> > >>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell > >>>>> wrote: > >>>>>> > >>>>>>> Mike, > >>>>>>> > >>>>>>> We have a delicate merge dance to perform. The disk_io_throttlin= g, > >>>>>>> solidfire, and object_store appear to have a number of overlappin= g > >>>>>>> elements. I understand the dependencies between the patches to b= e > as > >>>>>>> follows: > >>>>>>> > >>>>>>> object_store <- solidfire -> disk_io_throttling > >>>>>>> > >>>>>>> Am I correct that the device management aspects of SolidFire are > >>>>> additive > >>>>>>> to the object_store branch or there are circular dependency betwe= en > >>>>> the > >>>>>>> branches? Once we understand the dependency graph, we can > determine > >>>>> the > >>>>>>> best approach to land the changes in master. > >>>>>>> > >>>>>>> Thanks, > >>>>>>> -John > >>>>>>> > >>>>>>> > >>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski < > >>>>>> mike.tutkowski@solidfire.com> > >>>>>>> wrote: > >>>>>>> > >>>>>>>> Also, if we are good with Edison merging my code into his branch > >>>>> before > >>>>>>>> going into master, I am good with that. > >>>>>>>> > >>>>>>>> We can remove the StoragePoolType.Dynamic code after his merge a= nd > >>>>> we > >>>>>> can > >>>>>>>> deal with Burst IOPS then, as well. > >>>>>>>> > >>>>>>>> > >>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski < > >>>>>>>> mike.tutkowski@solidfire.com> wrote: > >>>>>>>> > >>>>>>>>> Let me make sure I follow where we're going here: > >>>>>>>>> > >>>>>>>>> 1) There should be NO references to hypervisor code in the > storage > >>>>>>>>> plug-ins code (this includes the default storage plug-in, which > >>>>>>> currently > >>>>>>>>> sends several commands to the hypervisor in use (although it do= es > >>>>> not > >>>>>>> know > >>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is actually in use)) > >>>>>>>>> > >>>>>>>>> 2) managed=3Dtrue or managed=3Dfalse can be placed in the url f= ield > (if > >>>>>> not > >>>>>>>>> present, we default to false). This info is stored in the > >>>>>>>>> storage_pool_details table. > >>>>>>>>> > >>>>>>>>> 3) When the "attach" command is sent to the hypervisor in > >>>>> question, we > >>>>>>>>> pass the managed property along (this takes the place of the > >>>>>>>>> StoragePoolType.Dynamic check). > >>>>>>>>> > >>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor checks for th= e > >>>>>> managed > >>>>>>>>> property. If true for an attach, the necessary hypervisor data > >>>>>>> structure is > >>>>>>>>> created and the rest of the attach command executes to attach t= he > >>>>>>> volume. > >>>>>>>>> > >>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to detach a > volume, > >>>>>> the > >>>>>>>>> same check is made. If managed, the hypervisor data structure i= s > >>>>>>> removed. > >>>>>>>>> > >>>>>>>>> 6) I do not see an clear way to support Burst IOPS in 4.2 unles= s > >>>>> it is > >>>>>>>>> stored in the volumes and disk_offerings table. If we have some > >>>>> idea, > >>>>>>>>> that'd be cool. > >>>>>>>>> > >>>>>>>>> Thanks! > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski < > >>>>>>>>> mike.tutkowski@solidfire.com> wrote: > >>>>>>>>> > >>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding > implementation > >>>>>>>>>> attributes. I always wondered about the details field. I thi= nk > >>>>> we > >>>>>>> should > >>>>>>>>>> beef up the description in the documentation regarding the > >>>>> expected > >>>>>>> format > >>>>>>>>>> of the field. In 4.1, I noticed that the details are not > >>>>> returned on > >>>>>>> the > >>>>>>>>>> createStoratePool updateStoragePool, or listStoragePool > response. > >>>>>> Why > >>>>>>>>>> don't we return it? It seems like it would be useful for > clients > >>>>> to > >>>>>> be > >>>>>>>>>> able to inspect the contents of the details field." > >>>>>>>>>> > >>>>>>>>>> Not sure how this would work storing Burst IOPS here. > >>>>>>>>>> > >>>>>>>>>> Burst IOPS need to be variable on a Disk Offering-by-Disk > Offering > >>>>>>>>>> basis. For each Disk Offering created, you have to be able to > >>>>>> associate > >>>>>>>>>> unique Burst IOPS. There is a disk_offering_details table. May= be > >>>>> it > >>>>>>> could > >>>>>>>>>> go there? > >>>>>>>>>> > >>>>>>>>>> I'm also not sure how you would accept the Burst IOPS in the G= UI > >>>>> if > >>>>>>> it's > >>>>>>>>>> not stored like the Min and Max fields are in the DB. > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> -- > >>>>>>>>> *Mike Tutkowski* > >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* > >>>>>>>>> e: mike.tutkowski@solidfire.com > >>>>>>>>> o: 303.746.7302 > >>>>>>>>> Advancing the way the world uses the cloud< > >>>>>>> http://solidfire.com/solution/overview/?video=3Dplay> > >>>>>>>>> *=99* > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> -- > >>>>>>>> *Mike Tutkowski* > >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* > >>>>>>>> e: mike.tutkowski@solidfire.com > >>>>>>>> o: 303.746.7302 > >>>>>>>> Advancing the way the world uses the > >>>>>>>> cloud > >>>>>>>> *=99* > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> *Mike Tutkowski* > >>>>>> *Senior CloudStack Developer, SolidFire Inc.* > >>>>>> e: mike.tutkowski@solidfire.com > >>>>>> o: 303.746.7302 > >>>>>> Advancing the way the world uses the > >>>>>> cloud > >>>>>> *=99* > >>>> > >>>> > >>>> > >>>> -- > >>>> *Mike Tutkowski* > >>>> *Senior CloudStack Developer, SolidFire Inc.* > >>>> e: mike.tutkowski@solidfire.com > >>>> o: 303.746.7302 > >>>> Advancing the way the world uses the cloud< > >> http://solidfire.com/solution/overview/?video=3Dplay> > >>>> *=99* > >>> > >>> > >>> > >>> -- > >>> *Mike Tutkowski* > >>> *Senior CloudStack Developer, SolidFire Inc.* > >>> e: mike.tutkowski@solidfire.com > >>> o: 303.746.7302 > >>> Advancing the way the world uses the > >>> cloud > >>> *=99* > > > > > > > > -- > > *Mike Tutkowski* > > *Senior CloudStack Developer, SolidFire Inc.* > > e: mike.tutkowski@solidfire.com > > o: 303.746.7302 > > Advancing the way the world uses the > > cloud > > *=99* > --=20 *Mike Tutkowski* *Senior CloudStack Developer, SolidFire Inc.* e: mike.tutkowski@solidfire.com o: 303.746.7302 Advancing the way the world uses the cloud *=99* --047d7b673d1afe82bf04def7f8c7--