Return-Path: X-Original-To: apmail-cloudstack-dev-archive@www.apache.org Delivered-To: apmail-cloudstack-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 0D4D911CFD for ; Thu, 12 Jun 2014 06:08:25 +0000 (UTC) Received: (qmail 61329 invoked by uid 500); 12 Jun 2014 06:08:24 -0000 Delivered-To: apmail-cloudstack-dev-archive@cloudstack.apache.org Received: (qmail 61283 invoked by uid 500); 12 Jun 2014 06:08:24 -0000 Mailing-List: contact dev-help@cloudstack.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cloudstack.apache.org Delivered-To: mailing list dev@cloudstack.apache.org Received: (qmail 61272 invoked by uid 99); 12 Jun 2014 06:08:24 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 12 Jun 2014 06:08:24 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of mike.tutkowski@solidfire.com designates 209.85.219.51 as permitted sender) Received: from [209.85.219.51] (HELO mail-oa0-f51.google.com) (209.85.219.51) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 12 Jun 2014 06:08:21 +0000 Received: by mail-oa0-f51.google.com with SMTP id j17so855286oag.38 for ; Wed, 11 Jun 2014 23:07:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=4qtOd3+QZLFXLrF4HlBNGxWlKSa/gCD5qJFdEHCJSbM=; b=jkjNNSBZ2BSRrGKarLedUN68O/I9vF4eJ/jjuNvIBpOwOyVcFq/0s6+El4dClv513N I0CKz6Y075+2UAPXqJYOV4/WPq11+yAqOWkf15hEemdAN5ID0Ux1n0d87jJZKeMVSrAE YoFSBBid5J7Lj9qdRlkqwbA5iGCvjMgPRrfWX4UIAOCHlx5+ozdFPoT5dmGqPdK6nnyj iEen8L/AxxxLIYhFjbVOu25MQaQLbdTMEZSa1S2Q97jKLb7a3pv4WT9n8hI/DJgt349X X2Lvs/lEQ/VcoCoPz3MKCvY8ZY/7OgZ09/DIzNzoaip2uiUZX9WPN4Hw3DA75uDRcdEd L7VQ== X-Gm-Message-State: ALoCoQnJUXdRhmJuEBYuz2JUnXt94Brbst/PIbWFYKqH1ieBPMnYxtLqkf3wca7SaTSZO2enWaMn MIME-Version: 1.0 X-Received: by 10.182.97.234 with SMTP id ed10mr41808631obb.31.1402553277095; Wed, 11 Jun 2014 23:07:57 -0700 (PDT) Received: by 10.182.73.197 with HTTP; Wed, 11 Jun 2014 23:07:57 -0700 (PDT) In-Reply-To: References: Date: Thu, 12 Jun 2014 00:07:57 -0600 Message-ID: Subject: Re: feature : changing volume properties dynamically in 4.5 From: Mike Tutkowski To: Punith S Cc: cloudstack , Amit Das , Kiran Mova Content-Type: multipart/alternative; boundary=047d7b2e4216dc84d204fb9d61bc X-Virus-Checked: Checked by ClamAV on apache.org --047d7b2e4216dc84d204fb9d61bc Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Sounds good - let me give some thought as to how we should break up the work. My GSoC student from Tunisia will be helping us, as well. On Wed, Jun 11, 2014 at 8:34 AM, Punith S wrote: > yes it sounds good, thanks for the proposal mike, > > yeah right now i have implemented prototype of my proposal, since its not > generic we shall implement your proposal for 4.5. > on the other hand, for 4.5 i'm supporting nfs protocol and resize feature > for managed storage in xenserver, now trying to implement snapshot and > support root disk for vm's. > and yes if we can club together, i can start working on this proposal for > data disk and get the prototype ready. > what do you think ? > > thanks. > > > On Wed, Jun 11, 2014 at 3:53 AM, Mike Tutkowski < > mike.tutkowski@solidfire.com> wrote: > >> I'll send out a [PROPOSAL] e-mail so others who may not be following thi= s >> e-mail thread have a better chance to comment on the feature. >> >> >> On Tue, Jun 10, 2014 at 2:58 PM, Mike Tutkowski < >> mike.tutkowski@solidfire.com> wrote: >> >>> Here is that design document I was referring to, Punith: >>> >>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=3D4256= 6111 >>> >>> I've been working with a student in Tunisia who is participating in >>> Google Summer of Code (GSoC) (I'm his mentor). >>> >>> He'll be working on part of this as will I. (He is also working on >>> another related task not listed here.) >>> >>> Feel free to join us, if you have time available, as we can divide out >>> coding and testing among the three of us. >>> >>> Talk to you later! >>> Mike >>> >>> >>> On Tue, Jun 10, 2014 at 10:18 AM, Mike Tutkowski < >>> mike.tutkowski@solidfire.com> wrote: >>> >>>> I plan to draw up a design document surrounding generic key/value pair= s >>>> today. >>>> >>>> Perhaps you can take a look at it when you have time, Punith? >>>> >>>> >>>> On Tue, Jun 10, 2014 at 10:06 AM, Mike Tutkowski < >>>> mike.tutkowski@solidfire.com> wrote: >>>> >>>>> Hi Punith, >>>>> >>>>> Yeah, I hear you about the number of permutations involved. >>>>> >>>>> Traditionally Compute and Disk Offerings have been immutable. It make= s >>>>> it easier from an accounting point of view for chargeback and billing= . >>>>> >>>>> You should definitely feel free to extend the CloudStack API. I think >>>>> NetApp did this for one of their storage features in the recent past.= This >>>>> way vendor-specific capabilities can be more easily offered without m= aking >>>>> it look like all vendors support those particular features. >>>>> >>>>> I do not yet have any code in master related to generic keys/values. >>>>> I'm still designing this. >>>>> >>>>> How does your schedule look for the 4.5 release? Do you think you >>>>> might have available cycles to help out with this generic key/value >>>>> implementation? >>>>> >>>>> Talk to you later! >>>>> Mike >>>>> >>>>> >>>>> On Tue, Jun 10, 2014 at 7:09 AM, Punith S >>>>> wrote: >>>>> >>>>>> hi mike, >>>>>> >>>>>> thanks for the reply, i like your approach which is a very generic >>>>>> way and also we only need to do a few changes to the current cloudst= ack, >>>>>> >>>>>> but on the other hand we are tying every property of the vendor to a >>>>>> disk offering through key/value pairs, since we offer lot of propert= ies >>>>>> like i mentioned, this can create a lot of permutation combinations = of disk >>>>>> offerings, for say if i need to turn deduplication On for a specific= volume >>>>>> , should i have to create a new disk offering having current propert= ies >>>>>> with deduplication On? >>>>>> >>>>>> is this approach already implemented in the current master ? >>>>>> >>>>>> and also like you mentioned about exposing a new api, is it okay if = i >>>>>> expose our own api in my util by extending the PlugableService like = in >>>>>> network plugins ? >>>>>> >>>>>> thanks. >>>>>> >>>>>> >>>>>> On Tue, Jun 10, 2014 at 1:17 AM, Mike Tutkowski < >>>>>> mike.tutkowski@solidfire.com> wrote: >>>>>> >>>>>>> Allow me to follow this up with more detail (with regards to what >>>>>>> Chris and I talked about): >>>>>>> >>>>>>> As you are aware, today the way you associate a Compute Offering >>>>>>> (CO) or a Disk Offering (DO) with a Primary Storage (PS) is via sto= rage >>>>>>> tagging. >>>>>>> >>>>>>> This has some benefits and drawbacks. >>>>>>> >>>>>>> One benefit is being able to have some level of vendor independence >>>>>>> from the point of view of the CO or DO. For example, if the storage= tag of >>>>>>> a DO is "Fast", then this can be satisfied by PS that describes its= elf as >>>>>>> "Fast", regardless of vendor. >>>>>>> >>>>>>> A major drawback with the storage-tagging approach, however, is tha= t >>>>>>> you are not easily able to leverage vendor-specific features, which= is >>>>>>> often why you bought storage from the vendor in question to begin w= ith. >>>>>>> >>>>>>> Ideally we do not want to add each vendor's features into the syste= m >>>>>>> as properties that can be seen by the admin regardless of whether o= r not >>>>>>> the underlying storage he's actually using supports the feature in = question. >>>>>>> >>>>>>> This coarse approach, however, was sort of business as usual when I >>>>>>> started in with CloudStack 1.5 years ago. >>>>>>> >>>>>>> That being the case, when I added QoS options to CS, I did so in a >>>>>>> way where the admin would see Min IOPS and Max IOPS options regardl= ess of >>>>>>> whether or not his storage actually supported those controls (to mi= tigate >>>>>>> this a bit in the GUI, the admin has to explicitly select "Storage = QoS" >>>>>>> from a combobox). >>>>>>> >>>>>>> We leverage the same use model with Hypervisor QoS: The admin sees >>>>>>> these options regardless of whether or not they actually apply on t= he >>>>>>> hypervisor where the VM gets deployed. >>>>>>> >>>>>>> Going forward, we want to implement a more fine-grain and generic >>>>>>> approach. >>>>>>> >>>>>>> We would like to have a storage provider field for the CO and DO >>>>>>> windows (this equates to the name of one and only one storage provi= der). If >>>>>>> the admin inputs a specific storage provider and does not use the s= torage >>>>>>> tags field, he can enter in an arbitrary number of key/value pairs = in >>>>>>> another text field (perhaps we would provide a nice entry dialog to= make >>>>>>> this easier in the GUI). These key value pairs can be passed into t= he >>>>>>> storage driver when it's asked to create (or update) a volume and t= he >>>>>>> storage driver can decide what each and every key/value pair means. >>>>>>> >>>>>>> What do you think about this approach? >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> >>>>>>> On Mon, Jun 9, 2014 at 1:14 PM, Mike Tutkowski < >>>>>>> mike.tutkowski@solidfire.com> wrote: >>>>>>> >>>>>>>> Hi Punith, >>>>>>>> >>>>>>>> This kind of a feature is something Chris Suich and I discussed a >>>>>>>> while back. >>>>>>>> >>>>>>>> We talked about leveraging arbitrary key/value pairs to make this >>>>>>>> happen (OpenStack does something similar). The key/value pairs wou= ld be >>>>>>>> vendor specific. >>>>>>>> >>>>>>>> If we take a key/value approach, we might be able to make this all >>>>>>>> work the way things work today when the user wants to change an ex= isting >>>>>>>> Compute Offering and/or Disk Offering. >>>>>>>> >>>>>>>> For example, the user would pick a new Compute Offering (with >>>>>>>> presumably has different key/value pairs) and CloudStack could inf= orm the >>>>>>>> applicable storage provider, who could update the volume in questi= on. >>>>>>>> >>>>>>>> This way we don't need to introduce a new API command and the use >>>>>>>> model for the user doesn't really change. >>>>>>>> >>>>>>>> What are you thoughts on this? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Mike >>>>>>>> >>>>>>>> >>>>>>>> On Mon, Jun 9, 2014 at 8:08 AM, Punith S >>>>>>>> wrote: >>>>>>>> >>>>>>>>> hi guys, >>>>>>>>> >>>>>>>>> since most of the third party storage providers have been >>>>>>>>> implementing 1:1 mapping(managed storage) between a volume(datase= t) and a >>>>>>>>> vm disk(vdi/vmdk) for guaranteeing the Qos, i would like to propo= se a new >>>>>>>>> feature to dynamically change the volume properties supported by = storage >>>>>>>>> vendors such as IOPS, Deduplication, Compression, Grace, Syncroni= zation, >>>>>>>>> Latency etc, depending on properties and features supported by re= spective >>>>>>>>> storage vendors. hence providing more flexibility for users. >>>>>>>>> >>>>>>>>> in case of using default cloudstack storage provider, we can >>>>>>>>> change the properties of the vdi/vmdk files apart from resizing t= he >>>>>>>>> volume(vdi/vmdk). >>>>>>>>> >>>>>>>>> changes in management server include, >>>>>>>>> >>>>>>>>> new async web api ChangeVolumePropertiesCmd, >>>>>>>>> new method in VolumeApiService for vo and dao validation >>>>>>>>> implementations. >>>>>>>>> new method in VolumeServiceManager for supporting callback and >>>>>>>>> calling the respective storage provider driver's implementation. >>>>>>>>> new method in PrimaryDataStoreDriver interface for implementing >>>>>>>>> respective features according to their storage product. >>>>>>>>> >>>>>>>>> changes in UI include, >>>>>>>>> new changing volume properties widget in volume section, showing >>>>>>>>> different properties depending upon listed storage providers. >>>>>>>>> >>>>>>>>> any suggestions and feedbacks ? >>>>>>>>> >>>>>>>>> thanks >>>>>>>>> >>>>>>>>> -- >>>>>>>>> regards, >>>>>>>>> >>>>>>>>> punith s >>>>>>>>> cloudbyte.com >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> *Mike Tutkowski* >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>>>> e: mike.tutkowski@solidfire.com >>>>>>>> o: 303.746.7302 >>>>>>>> Advancing the way the world uses the cloud >>>>>>>> *=E2=84=A2* >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> *Mike Tutkowski* >>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>>> e: mike.tutkowski@solidfire.com >>>>>>> o: 303.746.7302 >>>>>>> Advancing the way the world uses the cloud >>>>>>> *=E2=84=A2* >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> regards, >>>>>> >>>>>> punith s >>>>>> cloudbyte.com >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> *Mike Tutkowski* >>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>> e: mike.tutkowski@solidfire.com >>>>> o: 303.746.7302 >>>>> Advancing the way the world uses the cloud >>>>> *=E2=84=A2* >>>>> >>>> >>>> >>>> >>>> -- >>>> *Mike Tutkowski* >>>> *Senior CloudStack Developer, SolidFire Inc.* >>>> e: mike.tutkowski@solidfire.com >>>> o: 303.746.7302 >>>> Advancing the way the world uses the cloud >>>> *=E2=84=A2* >>>> >>> >>> >>> >>> -- >>> *Mike Tutkowski* >>> *Senior CloudStack Developer, SolidFire Inc.* >>> e: mike.tutkowski@solidfire.com >>> o: 303.746.7302 >>> Advancing the way the world uses the cloud >>> *=E2=84=A2* >>> >> >> >> >> -- >> *Mike Tutkowski* >> *Senior CloudStack Developer, SolidFire Inc.* >> e: mike.tutkowski@solidfire.com >> o: 303.746.7302 >> Advancing the way the world uses the cloud >> *=E2=84=A2* >> > > > > -- > regards, > > punith s > cloudbyte.com > --=20 *Mike Tutkowski* *Senior CloudStack Developer, SolidFire Inc.* e: mike.tutkowski@solidfire.com o: 303.746.7302 Advancing the way the world uses the cloud *=E2=84=A2* --047d7b2e4216dc84d204fb9d61bc--