Return-Path: X-Original-To: apmail-cloudstack-dev-archive@www.apache.org Delivered-To: apmail-cloudstack-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 0601F10E06 for ; Mon, 27 Jan 2014 19:26:39 +0000 (UTC) Received: (qmail 37728 invoked by uid 500); 27 Jan 2014 19:26:37 -0000 Delivered-To: apmail-cloudstack-dev-archive@cloudstack.apache.org Received: (qmail 37660 invoked by uid 500); 27 Jan 2014 19:26:37 -0000 Mailing-List: contact dev-help@cloudstack.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cloudstack.apache.org Delivered-To: mailing list dev@cloudstack.apache.org Received: (qmail 37651 invoked by uid 99); 27 Jan 2014 19:26:37 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 27 Jan 2014 19:26:37 +0000 X-ASF-Spam-Status: No, hits=-5.0 required=5.0 tests=RCVD_IN_DNSWL_HI,SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of Chris.Suich@netapp.com designates 216.240.18.77 as permitted sender) Received: from [216.240.18.77] (HELO mx12.netapp.com) (216.240.18.77) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 27 Jan 2014 19:26:31 +0000 X-IronPort-AV: E=Sophos;i="4.95,730,1384329600"; d="scan'208";a="139369312" Received: from vmwexceht04-prd.hq.netapp.com ([10.106.77.34]) by mx12-out.netapp.com with ESMTP; 27 Jan 2014 11:26:07 -0800 Received: from SACEXCMBX04-PRD.hq.netapp.com ([169.254.6.58]) by vmwexceht04-prd.hq.netapp.com ([10.106.77.34]) with mapi id 14.03.0123.003; Mon, 27 Jan 2014 11:26:07 -0800 From: "SuichII, Christopher" To: dev Subject: Re: Tags on storagePool Thread-Topic: Tags on storagePool Thread-Index: AQHO0C+I9GixrbomLEe33+Ro6Q6J0poCwweggAAfRgCAkgHhAIAAKbkAgAAB9ACAAAFdAIAAAKyAgAAA4YCAAFpggIAEV66AgAALOYCAAAIBgIAAMykAgAAKkACAAAE1gA== Date: Mon, 27 Jan 2014 19:26:05 +0000 Message-ID: <1DBD6B53-8CD9-470E-B108-C3DA3239ED13@netapp.com> References: <4BEFCC73-7B33-453A-BB5E-B6308C977EFE@netapp.com> <144DCBC7-2CE3-4ABD-AB19-6AC3418E08A0@netapp.com> <35957C18-53DF-42D1-8C3F-EDDA6B22741A@netapp.com> <93C65EE0-14C7-4405-8D9E-DD8CE1C5B9EA@netapp.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.106.53.51] Content-Type: text/plain; charset="Windows-1252" Content-ID: <9E6A0B90EC71D6468AF2C3ACAA0A22C5@hq.netapp.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Virus-Checked: Checked by ClamAV on apache.org I suspect the reason you don=92t have this problem is that you don=92t have= any storage pool details whose value is =91true=92. That is the only case = where this problem arises. I think we do have a problem differentiating, though. If you list all of th= e storage tags for a storage pool, then any storage pool details whose valu= e is =91true=92 will be assumed to be a storage tag. The only way around th= is problem I see is to cross reference the results there with all of the st= orage tags from disk offerings, and if you find a storage pool detail that = doesn=92t match a disk offering storage tag, then assume it isn=92t a stora= ge tag. Unfortunately it sounds like this is way out of scope for 4.3. I think we n= eed to start thinking about how we can fix this issue moving forward now th= at there will be plugins out there impacted by this. -Chris --=20 Chris Suich chris.suich@netapp.com NetApp Software Engineer Data Center Platforms =96 Cloud Solutions Citrix, Cisco & Red Hat On Jan 27, 2014, at 2:21 PM, Marcus Sorensen wrote: > I'm ok with whatever. It does seem like we have the ability to > differentiate real storage tags from details, since we can list/edit > those storage tags (I don't see all of my details showing up in the UI > when I look at the tags for my primary storage), though I'm not > immediately sure why or how at the moment. I think this affects Mike > the most, since he's shipping a plugin to customers. I can just look > at whatever changes are made and update my pools manually, if need be. >=20 > On Mon, Jan 27, 2014 at 11:43 AM, Mike Tutkowski > wrote: >> I agree...it certainly is not an ideal situation. >>=20 >> Have you assessed the risk involved with changing storage tags from usin= g >> true as a value to using something like storage_tag as a value? >>=20 >> If so, do you feel it is an acceptable amount of risk for 4.3 now that w= e >> have already begun spinning up RC builds? >>=20 >> Thanks >>=20 >>=20 >> On Mon, Jan 27, 2014 at 8:40 AM, SuichII, Christopher < >> Chris.Suich@netapp.com> wrote: >>=20 >>> I think my only real concern with working around it in this manner is t= hat >>> once a fix is applied, we now have a bunch of settings that do not conf= orm >>> to the true/false pattern. In order to migrate them from using t/f, yes= /no >>> or enabled/disabled back to using true/false, we would have to write so= me >>> kind of DB upgrade/migration in our plugin to update these values. In >>> addition, I would have to change the ConfigKey from being a boolean to = a >>> string and then back to a boolean once a solution is implemented. It is >>> just some technical debt that would pile up for plugins with a pretty n= asty >>> fix down the road. >>>=20 >>> -Chris >>> -- >>> Chris Suich >>> chris.suich@netapp.com >>> NetApp Software Engineer >>> Data Center Platforms =96 Cloud Solutions >>> Citrix, Cisco & Red Hat >>>=20 >>> On Jan 27, 2014, at 10:33 AM, Mike Tutkowski >>> wrote: >>>=20 >>>> It is a problem, but I think - at the moment - only NetApp, SolidFire, >>> and >>>> Marcus have storage plug-ins. >>>>=20 >>>> We could document this issue and the easy workaround of not using true >>> and >>>> false as a value and try to address it in 4.4. >>>>=20 >>>>=20 >>>> On Mon, Jan 27, 2014 at 7:53 AM, SuichII, Christopher < >>>> Chris.Suich@netapp.com> wrote: >>>>=20 >>>>> Any final thoughts on this? Letting this go into 4.3 could potentiall= y >>>>> cause issues with anyone using the configuration system for plugins i= n >>> 4.3. >>>>>=20 >>>>> -Chris >>>>> -- >>>>> Chris Suich >>>>> chris.suich@netapp.com >>>>> NetApp Software Engineer >>>>> Data Center Platforms =96 Cloud Solutions >>>>> Citrix, Cisco & Red Hat >>>>>=20 >>>>> On Jan 24, 2014, at 3:34 PM, SuichII, Christopher < >>> Chris.Suich@netapp.com> >>>>> wrote: >>>>>=20 >>>>>> This is the approach we=92ll have to take if we can=92t fix this in = 4.3. It >>>>> certainly works, but it isn=92t ideal. >>>>>>=20 >>>>>> Does anyone else have any thoughts on the impact this might have to >>> 4.3? >>>>>>=20 >>>>>> -Chris >>>>>> -- >>>>>> Chris Suich >>>>>> chris.suich@netapp.com >>>>>> NetApp Software Engineer >>>>>> Data Center Platforms =96 Cloud Solutions >>>>>> Citrix, Cisco & Red Hat >>>>>>=20 >>>>>> On Jan 24, 2014, at 10:11 AM, Mike Tutkowski < >>>>> mike.tutkowski@solidfire.com> wrote: >>>>>>=20 >>>>>>> This isn't a great solution, but you could also change the value fo= r >>>>> your >>>>>>> plug-in from true or false to something like t or f. >>>>>>>=20 >>>>>>>=20 >>>>>>> On Fri, Jan 24, 2014 at 8:08 AM, Mike Tutkowski < >>>>>>> mike.tutkowski@solidfire.com> wrote: >>>>>>>=20 >>>>>>>> Edison could confirm this perhaps, but I doubt any current >>> installation >>>>>>>> would have true for the value unless it was for a storage tag (the >>>>> plug-in >>>>>>>> framework just came out in 4.2). >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> On Fri, Jan 24, 2014 at 8:05 AM, Mike Tutkowski < >>>>>>>> mike.tutkowski@solidfire.com> wrote: >>>>>>>>=20 >>>>>>>>> I think your idea would be acceptable risk for 4.3. The upgrade >>> logic >>>>>>>>> would have to perform this true-to-storage_tag conversion, too, >>>>> though. >>>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>> On Fri, Jan 24, 2014 at 8:00 AM, SuichII, Christopher < >>>>>>>>> Chris.Suich@netapp.com> wrote: >>>>>>>>>=20 >>>>>>>>>> I think the quickest, easiest change would be to keep using the = tag >>>>> name >>>>>>>>>> as the key in the details table, but use a unique value like >>>>> =91storage_tag=92 >>>>>>>>>> instead of =91true=92. This wouldn=92t require any major logic c= hanges, >>>>> just the >>>>>>>>>> value used to check whether something is a storage tag. >>>>>>>>>>=20 >>>>>>>>>> It is quite risky for 4.3. However my concern is that if we let >>> this >>>>>>>>>> ship with 4.3, then any plugin that wants to use a storage pool >>>>> detail with >>>>>>>>>> the value =91true=92 will make updating the storage tag system M= UCH >>> more >>>>>>>>>> difficult. As far as I can tell, there is no other way to determ= ine >>>>> if >>>>>>>>>> something is a storage tag than checking if the details table va= lue >>>>> is >>>>>>>>>> =91true=92. If there are other details with the value =91true=92= , then we >>>>> wouldn=92t >>>>>>>>>> be able to differentiate between them for a DB upgrade between >>>>> versions. >>>>>>>>>>=20 >>>>>>>>>> -Chris >>>>>>>>>> -- >>>>>>>>>> Chris Suich >>>>>>>>>> chris.suich@netapp.com >>>>>>>>>> NetApp Software Engineer >>>>>>>>>> Data Center Platforms =96 Cloud Solutions >>>>>>>>>> Citrix, Cisco & Red Hat >>>>>>>>>>=20 >>>>>>>>>> On Jan 24, 2014, at 9:53 AM, Mike Tutkowski < >>>>>>>>>> mike.tutkowski@solidfire.com> wrote: >>>>>>>>>>=20 >>>>>>>>>>> I think at some point we need to use a key/value for storage ta= gs >>>>> such >>>>>>>>>> as >>>>>>>>>>> the following: >>>>>>>>>>>=20 >>>>>>>>>>> storageTags=3Dvalue1,value2,value3 >>>>>>>>>>>=20 >>>>>>>>>>> The problem with that is you have to parse the value cell each >>> time >>>>> you >>>>>>>>>>> pull it out of the DB. >>>>>>>>>>>=20 >>>>>>>>>>> That might be too risky of a change, though, for 4.3 at this >>> point. >>>>>>>>>>>=20 >>>>>>>>>>>=20 >>>>>>>>>>> On Fri, Jan 24, 2014 at 5:24 AM, SuichII, Christopher < >>>>>>>>>>> Chris.Suich@netapp.com> wrote: >>>>>>>>>>>=20 >>>>>>>>>>>> I have found an additional issue related to this. The allocato= rs >>> do >>>>>>>>>>>> properly ignore any storage pool details whose value is true t= hat >>>>> is >>>>>>>>>> not >>>>>>>>>>>> actually a storage pool. However, the list storage pools API d= oes >>>>>>>>>> NOT. When >>>>>>>>>>>> creating the StoragePoolResponse, it is still assumed that any >>>>>>>>>> storage pool >>>>>>>>>>>> detail with the value =91true=92 is a storage tag. >>>>>>>>>>>>=20 >>>>>>>>>>>> For my plugin targeting 4.3, we create a storage pool detail >>> with a >>>>>>>>>>>> true/false value, so this causes a problem with the storage po= ol >>>>> UI. >>>>>>>>>>>>=20 >>>>>>>>>>>> Any thoughts on how to fix this? >>>>>>>>>>>>=20 >>>>>>>>>>>> -Chris >>>>>>>>>>>> -- >>>>>>>>>>>> Chris Suich >>>>>>>>>>>> chris.suich@netapp.com >>>>>>>>>>>> NetApp Software Engineer >>>>>>>>>>>> Data Center Platforms =96 Cloud Solutions >>>>>>>>>>>> Citrix, Cisco & Red Hat >>>>>>>>>>>>=20 >>>>>>>>>>>> On Oct 23, 2013, at 6:43 PM, Alena Prokharchyk < >>>>>>>>>>>> Alena.Prokharchyk@citrix.com> wrote: >>>>>>>>>>>>=20 >>>>>>>>>>>>> Filed >>>>>>>>>>>>>=20 >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-4942 >>>>>>>>>>>>>=20 >>>>>>>>>>>>> -Alena. >>>>>>>>>>>>>=20 >>>>>>>>>>>>> From: Prachi Damle >>>>>>>>>>> Prachi.Damle@citrix.com>> >>>>>>>>>>>>> Date: Wednesday, October 23, 2013 2:04 PM >>>>>>>>>>>>> To: Alena Prokharchyk >>>>>>>>>>> alena.prokharchyk@citrix.com>>, "dev@cloudstack.apache.org >>> >>>>>>>>>>> dev@cloudstack.apache.org>" >>>>>>>>>>> dev@cloudstack.apache.org>> >>>>>>>>>>>>> Subject: RE: Tags on storagePool >>>>>>>>>>>>>=20 >>>>>>>>>>>>> Alena, >>>>>>>>>>>>>=20 >>>>>>>>>>>>> I don=92t know why it was designed this way in first place = =96 a >>>>> design >>>>>>>>>> like >>>>>>>>>>>> host_tags where we have separate table to store tags is much >>> better >>>>>>>>>> for >>>>>>>>>>>> Allocators to work on. >>>>>>>>>>>>>=20 >>>>>>>>>>>>> It is a bug, but will cause problem only if we land up with >>>>> situation >>>>>>>>>>>> explained below: >>>>>>>>>>>>>=20 >>>>>>>>>>>>> Given the existing design of storage tags, the Allocators sea= rch >>>>> the >>>>>>>>>>>> details table using the name =3D >>> and >>>>>>>>>> value >>>>>>>>>>>> =3Dtrue >>>>>>>>>>>>> Thus this will cause a problem in placement only if some othe= r >>>>>>>>>> storage >>>>>>>>>>>> pool detail happen to have the same =91name=92 as a storage-ta= g and >>>>> also a >>>>>>>>>>>> value =3D true. >>>>>>>>>>>>>=20 >>>>>>>>>>>>> -Prachi >>>>>>>>>>>>>=20 >>>>>>>>>>>>> From: Alena Prokharchyk >>>>>>>>>>>>> Sent: Wednesday, October 23, 2013 1:36 PM >>>>>>>>>>>>> To: Prachi Damle; dev@cloudstack.apache.org>>>>>>>>>>> dev@cloudstack.apache.org> >>>>>>>>>>>>> Subject: Tags on storagePool >>>>>>>>>>>>>=20 >>>>>>>>>>>>> I came across a potential bug in the way allocators do volume= s >>>>>>>>>> placement >>>>>>>>>>>> on storage, based on storage tags. Prachi, can you please conf= irm >>>>> if >>>>>>>>>> the >>>>>>>>>>>> bug is real. >>>>>>>>>>>>>=20 >>>>>>>>>>>>>=20 >>>>>>>>>>>>> The tags are passed to the createStoragePool API in form of >>>>>>>>>>>> tags=3D'tag1,tag2,..': >>>>>>>>>>>>>=20 >>>>>>>>>>>>> ?command=3DcreateStoragePool&...&tags=3Dalena >>>>>>>>>>>>>=20 >>>>>>>>>>>>> and stored in the storage_pool_details db table as: >>>>>>>>>>>>>=20 >>>>>>>>>>>>> mysql> select *from storage_pool_details where pool_id=3D2; >>>>>>>>>>>>> +----+---------+-----------+-------+ >>>>>>>>>>>>> | id | pool_id | name | value | >>>>>>>>>>>>> +----+---------+-----------+-------+ >>>>>>>>>>>>> | 2 | 2 | alenatags | true | >>>>>>>>>>>>> +----+---------+-----------+-------+ >>>>>>>>>>>>> 1 row in set (0.00 sec) >>>>>>>>>>>>>=20 >>>>>>>>>>>>>=20 >>>>>>>>>>>>> Allocator code assumes that everything stored in >>>>> storage_pool_details >>>>>>>>>>>> table, having value=3Dtrue - is a storage pool tag. And this i= s >>>>>>>>>> incorrect, as >>>>>>>>>>>> the storage_pool_details table is used for storing diff kinds = of >>>>>>>>>> storage >>>>>>>>>>>> pool details - not just tags - that can be later used by vario= us >>>>>>>>>> cloudStack >>>>>>>>>>>> managers. Like for example we save Storage level config >>> parameters >>>>> in >>>>>>>>>> this >>>>>>>>>>>> table ( >>>>>>>>>>>>=20 >>>>>>>>>>=20 >>>>>=20 >>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/FS+for+Granular+= Global+Configuration+Parameters >>>>>>>>>>>> ). >>>>>>>>>>>>>=20 >>>>>>>>>>>>>=20 >>>>>>>>>>>>> The correct way to fix it would be - store tags as: >>>>>>>>>>>>>=20 >>>>>>>>>>>>> +----+---------+-----------+-------+ >>>>>>>>>>>>> | id | pool_id | name | value | >>>>>>>>>>>>> +----+---------+-----------+-------+ >>>>>>>>>>>>> | 2 | 2 | tag | alena | >>>>>>>>>>>>> +----+---------+-----------+-------+ >>>>>>>>>>>>>=20 >>>>>>>>>>>>>=20 >>>>>>>>>>>>> and fix StorageManager to retrive all the tags by the "tag" k= ey. >>>>> We >>>>>>>>>> also >>>>>>>>>>>> have to fix the DB upgrade, which can be tricky as we will hav= e >>> to >>>>>>>>>> figure >>>>>>>>>>>> out which detail is a tag. >>>>>>>>>>>>>=20 >>>>>>>>>>>>>=20 >>>>>>>>>>>>> -Alena. >>>>>>>>>>>>>=20 >>>>>>>>>>>>=20 >>>>>>>>>>>>=20 >>>>>>>>>>>=20 >>>>>>>>>>>=20 >>>>>>>>>>> -- >>>>>>>>>>> *Mike Tutkowski* >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>>>>>>> e: mike.tutkowski@solidfire.com >>>>>>>>>>> o: 303.746.7302 >>>>>>>>>>> Advancing the way the world uses the >>>>>>>>>>> cloud >>>>>>>>>>> *=99* >>>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>> -- >>>>>>>>> *Mike Tutkowski* >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>>>>> e: mike.tutkowski@solidfire.com >>>>>>>>> o: 303.746.7302 >>>>>>>>> Advancing the way the world uses the cloud< >>>>> http://solidfire.com/solution/overview/?video=3Dplay> >>>>>>>>> *=99* >>>>>>>>>=20 >>>>>>>>=20 >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> -- >>>>>>>> *Mike Tutkowski* >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>>>> e: mike.tutkowski@solidfire.com >>>>>>>> o: 303.746.7302 >>>>>>>> Advancing the way the world uses the cloud< >>>>> http://solidfire.com/solution/overview/?video=3Dplay> >>>>>>>> *=99* >>>>>>>>=20 >>>>>>>=20 >>>>>>>=20 >>>>>>>=20 >>>>>>> -- >>>>>>> *Mike Tutkowski* >>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>>> e: mike.tutkowski@solidfire.com >>>>>>> o: 303.746.7302 >>>>>>> Advancing the way the world uses the >>>>>>> cloud >>>>>>> *=99* >>>>>>=20 >>>>>=20 >>>>>=20 >>>>=20 >>>>=20 >>>> -- >>>> *Mike Tutkowski* >>>> *Senior CloudStack Developer, SolidFire Inc.* >>>> e: mike.tutkowski@solidfire.com >>>> o: 303.746.7302 >>>> Advancing the way the world uses the >>>> cloud >>>> *=99* >>>=20 >>>=20 >>=20 >>=20 >> -- >> *Mike Tutkowski* >> *Senior CloudStack Developer, SolidFire Inc.* >> e: mike.tutkowski@solidfire.com >> o: 303.746.7302 >> Advancing the way the world uses the >> cloud >> *=99*