Return-Path: X-Original-To: apmail-cloudstack-dev-archive@www.apache.org Delivered-To: apmail-cloudstack-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 8808710E5B for ; Mon, 27 Jan 2014 19:32:35 +0000 (UTC) Received: (qmail 57427 invoked by uid 500); 27 Jan 2014 19:32:34 -0000 Delivered-To: apmail-cloudstack-dev-archive@cloudstack.apache.org Received: (qmail 57393 invoked by uid 500); 27 Jan 2014 19:32:34 -0000 Mailing-List: contact dev-help@cloudstack.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cloudstack.apache.org Delivered-To: mailing list dev@cloudstack.apache.org Received: (qmail 57378 invoked by uid 99); 27 Jan 2014 19:32:33 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 27 Jan 2014 19:32:33 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of shadowsor@gmail.com designates 209.85.215.49 as permitted sender) Received: from [209.85.215.49] (HELO mail-la0-f49.google.com) (209.85.215.49) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 27 Jan 2014 19:32:30 +0000 Received: by mail-la0-f49.google.com with SMTP id y1so4762405lam.22 for ; Mon, 27 Jan 2014 11:32:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; bh=fx6tTeAK7CA2apgAFkOTWU7jprQE54yUmQDpCOjukPg=; b=WbdUDgUYcRvup013O4ePFZbLSIRNFtl5F9q2sR8Bm5Q+6A5eNq4zrtWOP4W7R6UUJo hVx3UXxK77fyQSA3WDsE3fQyTmZSXV2lTom5Sj8sOxL7hPXbnkg0WUv77eAaZsy8E0ao XZnLmI+vbTH7WXt6zvafpZN0qK99ZhZy9OtsR7f40uJiCoOtO8bxjSp+YYqvmfLjnEfE QdD3JGT86zAYlPkm2Tea7YvjOUvEj+VlKYw2x597hjmm7nfUdtn3sgQ3Q+4yfJQW1Gwc lGnPccn5q4Yl+OXwwhdWS0YUQHGqI85Ed1LrAXyQl9y3yr3oGepg5rUIlnwcUUIsix6q UPig== MIME-Version: 1.0 X-Received: by 10.153.7.137 with SMTP id dc9mr18798544lad.25.1390851128310; Mon, 27 Jan 2014 11:32:08 -0800 (PST) Received: by 10.114.18.143 with HTTP; Mon, 27 Jan 2014 11:32:08 -0800 (PST) In-Reply-To: References: <4BEFCC73-7B33-453A-BB5E-B6308C977EFE@netapp.com> <144DCBC7-2CE3-4ABD-AB19-6AC3418E08A0@netapp.com> <35957C18-53DF-42D1-8C3F-EDDA6B22741A@netapp.com> <93C65EE0-14C7-4405-8D9E-DD8CE1C5B9EA@netapp.com> <1DBD6B53-8CD9-470E-B108-C3DA3239ED13@netapp.com> Date: Mon, 27 Jan 2014 12:32:08 -0700 Message-ID: Subject: Re: Tags on storagePool From: Marcus Sorensen To: "dev@cloudstack.apache.org" Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org Right, for some reason I was thinking in the moment that all details would have a value of 'true'. Since that's not the case, and this field is clearly not a boolean field, we should just change them to storage_tag for a value, as suggested. On Mon, Jan 27, 2014 at 12:28 PM, Mike Tutkowski wrote: > I agree...we should fix it in 4.4. > > Would you write up a JIRA ticket for it, Chris? > > Thanks! > > > On Mon, Jan 27, 2014 at 12:26 PM, SuichII, Christopher < > Chris.Suich@netapp.com> wrote: > >> I suspect the reason you don=92t have this problem is that you don=92t h= ave >> any storage pool details whose value is =91true=92. That is the only cas= e where >> this problem arises. >> >> I think we do have a problem differentiating, though. If you list all of >> the storage tags for a storage pool, then any storage pool details whose >> value is =91true=92 will be assumed to be a storage tag. The only way ar= ound >> this problem I see is to cross reference the results there with all of t= he >> storage tags from disk offerings, and if you find a storage pool detail >> that doesn=92t match a disk offering storage tag, then assume it isn=92t= a >> storage tag. >> >> Unfortunately it sounds like this is way out of scope for 4.3. I think w= e >> need to start thinking about how we can fix this issue moving forward no= w >> that there will be plugins out there impacted by this. >> >> -Chris >> -- >> Chris Suich >> chris.suich@netapp.com >> NetApp Software Engineer >> Data Center Platforms =96 Cloud Solutions >> Citrix, Cisco & Red Hat >> >> On Jan 27, 2014, at 2:21 PM, Marcus Sorensen wrote= : >> >> > I'm ok with whatever. It does seem like we have the ability to >> > differentiate real storage tags from details, since we can list/edit >> > those storage tags (I don't see all of my details showing up in the UI >> > when I look at the tags for my primary storage), though I'm not >> > immediately sure why or how at the moment. I think this affects Mike >> > the most, since he's shipping a plugin to customers. I can just look >> > at whatever changes are made and update my pools manually, if need be. >> > >> > On Mon, Jan 27, 2014 at 11:43 AM, Mike Tutkowski >> > wrote: >> >> I agree...it certainly is not an ideal situation. >> >> >> >> Have you assessed the risk involved with changing storage tags from >> using >> >> true as a value to using something like storage_tag as a value? >> >> >> >> If so, do you feel it is an acceptable amount of risk for 4.3 now tha= t >> we >> >> have already begun spinning up RC builds? >> >> >> >> Thanks >> >> >> >> >> >> On Mon, Jan 27, 2014 at 8:40 AM, SuichII, Christopher < >> >> Chris.Suich@netapp.com> wrote: >> >> >> >>> I think my only real concern with working around it in this manner i= s >> that >> >>> once a fix is applied, we now have a bunch of settings that do not >> conform >> >>> to the true/false pattern. In order to migrate them from using t/f, >> yes/no >> >>> or enabled/disabled back to using true/false, we would have to write >> some >> >>> kind of DB upgrade/migration in our plugin to update these values. I= n >> >>> addition, I would have to change the ConfigKey from being a boolean = to >> a >> >>> string and then back to a boolean once a solution is implemented. It= is >> >>> just some technical debt that would pile up for plugins with a prett= y >> nasty >> >>> fix down the road. >> >>> >> >>> -Chris >> >>> -- >> >>> Chris Suich >> >>> chris.suich@netapp.com >> >>> NetApp Software Engineer >> >>> Data Center Platforms =96 Cloud Solutions >> >>> Citrix, Cisco & Red Hat >> >>> >> >>> On Jan 27, 2014, at 10:33 AM, Mike Tutkowski < >> mike.tutkowski@solidfire.com> >> >>> wrote: >> >>> >> >>>> It is a problem, but I think - at the moment - only NetApp, SolidFi= re, >> >>> and >> >>>> Marcus have storage plug-ins. >> >>>> >> >>>> We could document this issue and the easy workaround of not using t= rue >> >>> and >> >>>> false as a value and try to address it in 4.4. >> >>>> >> >>>> >> >>>> On Mon, Jan 27, 2014 at 7:53 AM, SuichII, Christopher < >> >>>> Chris.Suich@netapp.com> wrote: >> >>>> >> >>>>> Any final thoughts on this? Letting this go into 4.3 could >> potentially >> >>>>> cause issues with anyone using the configuration system for plugin= s >> in >> >>> 4.3. >> >>>>> >> >>>>> -Chris >> >>>>> -- >> >>>>> Chris Suich >> >>>>> chris.suich@netapp.com >> >>>>> NetApp Software Engineer >> >>>>> Data Center Platforms =96 Cloud Solutions >> >>>>> Citrix, Cisco & Red Hat >> >>>>> >> >>>>> On Jan 24, 2014, at 3:34 PM, SuichII, Christopher < >> >>> Chris.Suich@netapp.com> >> >>>>> wrote: >> >>>>> >> >>>>>> This is the approach we=92ll have to take if we can=92t fix this = in >> 4.3. It >> >>>>> certainly works, but it isn=92t ideal. >> >>>>>> >> >>>>>> Does anyone else have any thoughts on the impact this might have = to >> >>> 4.3? >> >>>>>> >> >>>>>> -Chris >> >>>>>> -- >> >>>>>> Chris Suich >> >>>>>> chris.suich@netapp.com >> >>>>>> NetApp Software Engineer >> >>>>>> Data Center Platforms =96 Cloud Solutions >> >>>>>> Citrix, Cisco & Red Hat >> >>>>>> >> >>>>>> On Jan 24, 2014, at 10:11 AM, Mike Tutkowski < >> >>>>> mike.tutkowski@solidfire.com> wrote: >> >>>>>> >> >>>>>>> This isn't a great solution, but you could also change the value >> for >> >>>>> your >> >>>>>>> plug-in from true or false to something like t or f. >> >>>>>>> >> >>>>>>> >> >>>>>>> On Fri, Jan 24, 2014 at 8:08 AM, Mike Tutkowski < >> >>>>>>> mike.tutkowski@solidfire.com> wrote: >> >>>>>>> >> >>>>>>>> Edison could confirm this perhaps, but I doubt any current >> >>> installation >> >>>>>>>> would have true for the value unless it was for a storage tag (= the >> >>>>> plug-in >> >>>>>>>> framework just came out in 4.2). >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> On Fri, Jan 24, 2014 at 8:05 AM, Mike Tutkowski < >> >>>>>>>> mike.tutkowski@solidfire.com> wrote: >> >>>>>>>> >> >>>>>>>>> I think your idea would be acceptable risk for 4.3. The upgrad= e >> >>> logic >> >>>>>>>>> would have to perform this true-to-storage_tag conversion, too= , >> >>>>> though. >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> On Fri, Jan 24, 2014 at 8:00 AM, SuichII, Christopher < >> >>>>>>>>> Chris.Suich@netapp.com> wrote: >> >>>>>>>>> >> >>>>>>>>>> I think the quickest, easiest change would be to keep using t= he >> tag >> >>>>> name >> >>>>>>>>>> as the key in the details table, but use a unique value like >> >>>>> =91storage_tag=92 >> >>>>>>>>>> instead of =91true=92. This wouldn=92t require any major logi= c >> changes, >> >>>>> just the >> >>>>>>>>>> value used to check whether something is a storage tag. >> >>>>>>>>>> >> >>>>>>>>>> It is quite risky for 4.3. However my concern is that if we l= et >> >>> this >> >>>>>>>>>> ship with 4.3, then any plugin that wants to use a storage po= ol >> >>>>> detail with >> >>>>>>>>>> the value =91true=92 will make updating the storage tag syste= m MUCH >> >>> more >> >>>>>>>>>> difficult. As far as I can tell, there is no other way to >> determine >> >>>>> if >> >>>>>>>>>> something is a storage tag than checking if the details table >> value >> >>>>> is >> >>>>>>>>>> =91true=92. If there are other details with the value =91true= =92, then >> we >> >>>>> wouldn=92t >> >>>>>>>>>> be able to differentiate between them for a DB upgrade betwee= n >> >>>>> versions. >> >>>>>>>>>> >> >>>>>>>>>> -Chris >> >>>>>>>>>> -- >> >>>>>>>>>> Chris Suich >> >>>>>>>>>> chris.suich@netapp.com >> >>>>>>>>>> NetApp Software Engineer >> >>>>>>>>>> Data Center Platforms =96 Cloud Solutions >> >>>>>>>>>> Citrix, Cisco & Red Hat >> >>>>>>>>>> >> >>>>>>>>>> On Jan 24, 2014, at 9:53 AM, Mike Tutkowski < >> >>>>>>>>>> mike.tutkowski@solidfire.com> wrote: >> >>>>>>>>>> >> >>>>>>>>>>> I think at some point we need to use a key/value for storage >> tags >> >>>>> such >> >>>>>>>>>> as >> >>>>>>>>>>> the following: >> >>>>>>>>>>> >> >>>>>>>>>>> storageTags=3Dvalue1,value2,value3 >> >>>>>>>>>>> >> >>>>>>>>>>> The problem with that is you have to parse the value cell ea= ch >> >>> time >> >>>>> you >> >>>>>>>>>>> pull it out of the DB. >> >>>>>>>>>>> >> >>>>>>>>>>> That might be too risky of a change, though, for 4.3 at this >> >>> point. >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> On Fri, Jan 24, 2014 at 5:24 AM, SuichII, Christopher < >> >>>>>>>>>>> Chris.Suich@netapp.com> wrote: >> >>>>>>>>>>> >> >>>>>>>>>>>> I have found an additional issue related to this. The >> allocators >> >>> do >> >>>>>>>>>>>> properly ignore any storage pool details whose value is tru= e >> that >> >>>>> is >> >>>>>>>>>> not >> >>>>>>>>>>>> actually a storage pool. However, the list storage pools AP= I >> does >> >>>>>>>>>> NOT. When >> >>>>>>>>>>>> creating the StoragePoolResponse, it is still assumed that = any >> >>>>>>>>>> storage pool >> >>>>>>>>>>>> detail with the value =91true=92 is a storage tag. >> >>>>>>>>>>>> >> >>>>>>>>>>>> For my plugin targeting 4.3, we create a storage pool detai= l >> >>> with a >> >>>>>>>>>>>> true/false value, so this causes a problem with the storage >> pool >> >>>>> UI. >> >>>>>>>>>>>> >> >>>>>>>>>>>> Any thoughts on how to fix this? >> >>>>>>>>>>>> >> >>>>>>>>>>>> -Chris >> >>>>>>>>>>>> -- >> >>>>>>>>>>>> Chris Suich >> >>>>>>>>>>>> chris.suich@netapp.com >> >>>>>>>>>>>> NetApp Software Engineer >> >>>>>>>>>>>> Data Center Platforms =96 Cloud Solutions >> >>>>>>>>>>>> Citrix, Cisco & Red Hat >> >>>>>>>>>>>> >> >>>>>>>>>>>> On Oct 23, 2013, at 6:43 PM, Alena Prokharchyk < >> >>>>>>>>>>>> Alena.Prokharchyk@citrix.com> wrote: >> >>>>>>>>>>>> >> >>>>>>>>>>>>> Filed >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-4942 >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> -Alena. >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> From: Prachi Damle > >>>>>>>>>>>> Prachi.Damle@citrix.com>> >> >>>>>>>>>>>>> Date: Wednesday, October 23, 2013 2:04 PM >> >>>>>>>>>>>>> To: Alena Prokharchyk > >>>>>>>>>>>> alena.prokharchyk@citrix.com>>, "dev@cloudstack.apache.org >> >>> > >>>>>>>>>>>> dev@cloudstack.apache.org>" > > >>>>>>>>>>>> dev@cloudstack.apache.org>> >> >>>>>>>>>>>>> Subject: RE: Tags on storagePool >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> Alena, >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> I don=92t know why it was designed this way in first place= =96 a >> >>>>> design >> >>>>>>>>>> like >> >>>>>>>>>>>> host_tags where we have separate table to store tags is muc= h >> >>> better >> >>>>>>>>>> for >> >>>>>>>>>>>> Allocators to work on. >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> It is a bug, but will cause problem only if we land up wit= h >> >>>>> situation >> >>>>>>>>>>>> explained below: >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> Given the existing design of storage tags, the Allocators >> search >> >>>>> the >> >>>>>>>>>>>> details table using the name =3D >> >>> and >> >>>>>>>>>> value >> >>>>>>>>>>>> =3Dtrue >> >>>>>>>>>>>>> Thus this will cause a problem in placement only if some >> other >> >>>>>>>>>> storage >> >>>>>>>>>>>> pool detail happen to have the same =91name=92 as a storage= -tag >> and >> >>>>> also a >> >>>>>>>>>>>> value =3D true. >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> -Prachi >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> From: Alena Prokharchyk >> >>>>>>>>>>>>> Sent: Wednesday, October 23, 2013 1:36 PM >> >>>>>>>>>>>>> To: Prachi Damle; dev@cloudstack.apache.org> >>>>>>>>>>>> dev@cloudstack.apache.org> >> >>>>>>>>>>>>> Subject: Tags on storagePool >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> I came across a potential bug in the way allocators do >> volumes >> >>>>>>>>>> placement >> >>>>>>>>>>>> on storage, based on storage tags. Prachi, can you please >> confirm >> >>>>> if >> >>>>>>>>>> the >> >>>>>>>>>>>> bug is real. >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> The tags are passed to the createStoragePool API in form o= f >> >>>>>>>>>>>> tags=3D'tag1,tag2,..': >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> ?command=3DcreateStoragePool&...&tags=3Dalena >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> and stored in the storage_pool_details db table as: >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> mysql> select *from storage_pool_details where pool_id=3D2= ; >> >>>>>>>>>>>>> +----+---------+-----------+-------+ >> >>>>>>>>>>>>> | id | pool_id | name | value | >> >>>>>>>>>>>>> +----+---------+-----------+-------+ >> >>>>>>>>>>>>> | 2 | 2 | alenatags | true | >> >>>>>>>>>>>>> +----+---------+-----------+-------+ >> >>>>>>>>>>>>> 1 row in set (0.00 sec) >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> Allocator code assumes that everything stored in >> >>>>> storage_pool_details >> >>>>>>>>>>>> table, having value=3Dtrue - is a storage pool tag. And thi= s is >> >>>>>>>>>> incorrect, as >> >>>>>>>>>>>> the storage_pool_details table is used for storing diff kin= ds >> of >> >>>>>>>>>> storage >> >>>>>>>>>>>> pool details - not just tags - that can be later used by >> various >> >>>>>>>>>> cloudStack >> >>>>>>>>>>>> managers. Like for example we save Storage level config >> >>> parameters >> >>>>> in >> >>>>>>>>>> this >> >>>>>>>>>>>> table ( >> >>>>>>>>>>>> >> >>>>>>>>>> >> >>>>> >> >>> >> https://cwiki.apache.org/confluence/display/CLOUDSTACK/FS+for+Granular+G= lobal+Configuration+Parameters >> >>>>>>>>>>>> ). >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> The correct way to fix it would be - store tags as: >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> +----+---------+-----------+-------+ >> >>>>>>>>>>>>> | id | pool_id | name | value | >> >>>>>>>>>>>>> +----+---------+-----------+-------+ >> >>>>>>>>>>>>> | 2 | 2 | tag | alena | >> >>>>>>>>>>>>> +----+---------+-----------+-------+ >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> and fix StorageManager to retrive all the tags by the "tag= " >> key. >> >>>>> We >> >>>>>>>>>> also >> >>>>>>>>>>>> have to fix the DB upgrade, which can be tricky as we will >> have >> >>> to >> >>>>>>>>>> figure >> >>>>>>>>>>>> out which detail is a tag. >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> -Alena. >> >>>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> -- >> >>>>>>>>>>> *Mike Tutkowski* >> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >> >>>>>>>>>>> e: mike.tutkowski@solidfire.com >> >>>>>>>>>>> o: 303.746.7302 >> >>>>>>>>>>> Advancing the way the world uses the >> >>>>>>>>>>> cloud >> >>>>>>>>>>> *=99* >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> -- >> >>>>>>>>> *Mike Tutkowski* >> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >> >>>>>>>>> e: mike.tutkowski@solidfire.com >> >>>>>>>>> o: 303.746.7302 >> >>>>>>>>> Advancing the way the world uses the cloud< >> >>>>> http://solidfire.com/solution/overview/?video=3Dplay> >> >>>>>>>>> *=99* >> >>>>>>>>> >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> -- >> >>>>>>>> *Mike Tutkowski* >> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >> >>>>>>>> e: mike.tutkowski@solidfire.com >> >>>>>>>> o: 303.746.7302 >> >>>>>>>> Advancing the way the world uses the cloud< >> >>>>> http://solidfire.com/solution/overview/?video=3Dplay> >> >>>>>>>> *=99* >> >>>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> -- >> >>>>>>> *Mike Tutkowski* >> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >> >>>>>>> e: mike.tutkowski@solidfire.com >> >>>>>>> o: 303.746.7302 >> >>>>>>> Advancing the way the world uses the >> >>>>>>> cloud >> >>>>>>> *=99* >> >>>>>> >> >>>>> >> >>>>> >> >>>> >> >>>> >> >>>> -- >> >>>> *Mike Tutkowski* >> >>>> *Senior CloudStack Developer, SolidFire Inc.* >> >>>> e: mike.tutkowski@solidfire.com >> >>>> o: 303.746.7302 >> >>>> Advancing the way the world uses the >> >>>> cloud >> >>>> *=99* >> >>> >> >>> >> >> >> >> >> >> -- >> >> *Mike Tutkowski* >> >> *Senior CloudStack Developer, SolidFire Inc.* >> >> e: mike.tutkowski@solidfire.com >> >> o: 303.746.7302 >> >> Advancing the way the world uses the >> >> cloud >> >> *=99* >> >> > > > -- > *Mike Tutkowski* > *Senior CloudStack Developer, SolidFire Inc.* > e: mike.tutkowski@solidfire.com > o: 303.746.7302 > Advancing the way the world uses the > cloud > *=99*