cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "SuichII, Christopher" <Chris.Su...@netapp.com>
Subject Re: Tags on storagePool
Date Mon, 27 Jan 2014 19:26:05 GMT
I suspect the reason you don’t have this problem is that you don’t have any storage pool
details whose value is ‘true’. That is the only case where this problem arises.

I think we do have a problem differentiating, though. If you list all of the storage tags
for a storage pool, then any storage pool details whose value is ‘true’ will be assumed
to be a storage tag. The only way around this problem I see is to cross reference the results
there with all of the storage tags from disk offerings, and if you find a storage pool detail
that doesn’t match a disk offering storage tag, then assume it isn’t a storage tag.

Unfortunately it sounds like this is way out of scope for 4.3. I think we need to start thinking
about how we can fix this issue moving forward now that there will be plugins out there impacted
by this.

-Chris
-- 
Chris Suich
chris.suich@netapp.com
NetApp Software Engineer
Data Center Platforms – Cloud Solutions
Citrix, Cisco & Red Hat

On Jan 27, 2014, at 2:21 PM, Marcus Sorensen <shadowsor@gmail.com> wrote:

> I'm ok with whatever. It does seem like we have the ability to
> differentiate real storage tags from details, since we can list/edit
> those storage tags (I don't see all of my details showing up in the UI
> when I look at the tags for my primary storage), though I'm not
> immediately sure why or how at the moment. I think this affects Mike
> the most, since he's shipping a plugin to customers. I can just look
> at whatever changes are made and update my pools manually, if need be.
> 
> On Mon, Jan 27, 2014 at 11:43 AM, Mike Tutkowski
> <mike.tutkowski@solidfire.com> wrote:
>> I agree...it certainly is not an ideal situation.
>> 
>> Have you assessed the risk involved with changing storage tags from using
>> true as a value to using something like storage_tag as a value?
>> 
>> If so, do you feel it is an acceptable amount of risk for 4.3 now that we
>> have already begun spinning up RC builds?
>> 
>> Thanks
>> 
>> 
>> On Mon, Jan 27, 2014 at 8:40 AM, SuichII, Christopher <
>> Chris.Suich@netapp.com> wrote:
>> 
>>> I think my only real concern with working around it in this manner is that
>>> once a fix is applied, we now have a bunch of settings that do not conform
>>> to the true/false pattern. In order to migrate them from using t/f, yes/no
>>> or enabled/disabled back to using true/false, we would have to write some
>>> kind of DB upgrade/migration in our plugin to update these values. In
>>> addition, I would have to change the ConfigKey from being a boolean to a
>>> string and then back to a boolean once a solution is implemented. It is
>>> just some technical debt that would pile up for plugins with a pretty nasty
>>> fix down the road.
>>> 
>>> -Chris
>>> --
>>> Chris Suich
>>> chris.suich@netapp.com
>>> NetApp Software Engineer
>>> Data Center Platforms – Cloud Solutions
>>> Citrix, Cisco & Red Hat
>>> 
>>> On Jan 27, 2014, at 10:33 AM, Mike Tutkowski <mike.tutkowski@solidfire.com>
>>> wrote:
>>> 
>>>> It is a problem, but I think - at the moment - only NetApp, SolidFire,
>>> and
>>>> Marcus have storage plug-ins.
>>>> 
>>>> We could document this issue and the easy workaround of not using true
>>> and
>>>> false as a value and try to address it in 4.4.
>>>> 
>>>> 
>>>> On Mon, Jan 27, 2014 at 7:53 AM, SuichII, Christopher <
>>>> Chris.Suich@netapp.com> wrote:
>>>> 
>>>>> Any final thoughts on this? Letting this go into 4.3 could potentially
>>>>> cause issues with anyone using the configuration system for plugins in
>>> 4.3.
>>>>> 
>>>>> -Chris
>>>>> --
>>>>> Chris Suich
>>>>> chris.suich@netapp.com
>>>>> NetApp Software Engineer
>>>>> Data Center Platforms – Cloud Solutions
>>>>> Citrix, Cisco & Red Hat
>>>>> 
>>>>> On Jan 24, 2014, at 3:34 PM, SuichII, Christopher <
>>> Chris.Suich@netapp.com>
>>>>> wrote:
>>>>> 
>>>>>> This is the approach we’ll have to take if we can’t fix this
in 4.3. It
>>>>> certainly works, but it isn’t ideal.
>>>>>> 
>>>>>> Does anyone else have any thoughts on the impact this might have
to
>>> 4.3?
>>>>>> 
>>>>>> -Chris
>>>>>> --
>>>>>> Chris Suich
>>>>>> chris.suich@netapp.com
>>>>>> NetApp Software Engineer
>>>>>> Data Center Platforms – Cloud Solutions
>>>>>> Citrix, Cisco & Red Hat
>>>>>> 
>>>>>> On Jan 24, 2014, at 10:11 AM, Mike Tutkowski <
>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>> 
>>>>>>> This isn't a great solution, but you could also change the value
for
>>>>> your
>>>>>>> plug-in from true or false to something like t or f.
>>>>>>> 
>>>>>>> 
>>>>>>> On Fri, Jan 24, 2014 at 8:08 AM, Mike Tutkowski <
>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>> 
>>>>>>>> Edison could confirm this perhaps, but I doubt any current
>>> installation
>>>>>>>> would have true for the value unless it was for a storage
tag (the
>>>>> plug-in
>>>>>>>> framework just came out in 4.2).
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Fri, Jan 24, 2014 at 8:05 AM, Mike Tutkowski <
>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>> 
>>>>>>>>> I think your idea would be acceptable risk for 4.3. The
upgrade
>>> logic
>>>>>>>>> would have to perform this true-to-storage_tag conversion,
too,
>>>>> though.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On Fri, Jan 24, 2014 at 8:00 AM, SuichII, Christopher
<
>>>>>>>>> Chris.Suich@netapp.com> wrote:
>>>>>>>>> 
>>>>>>>>>> I think the quickest, easiest change would be to
keep using the tag
>>>>> name
>>>>>>>>>> as the key in the details table, but use a unique
value like
>>>>> ‘storage_tag’
>>>>>>>>>> instead of ‘true’. This wouldn’t require any
major logic changes,
>>>>> just the
>>>>>>>>>> value used to check whether something is a storage
tag.
>>>>>>>>>> 
>>>>>>>>>> It is quite risky for 4.3. However my concern is
that if we let
>>> this
>>>>>>>>>> ship with 4.3, then any plugin that wants to use
a storage pool
>>>>> detail with
>>>>>>>>>> the value ‘true’ will make updating the storage
tag system MUCH
>>> more
>>>>>>>>>> difficult. As far as I can tell, there is no other
way to determine
>>>>> if
>>>>>>>>>> something is a storage tag than checking if the details
table value
>>>>> is
>>>>>>>>>> ‘true’. If there are other details with the value
‘true’, then we
>>>>> wouldn’t
>>>>>>>>>> be able to differentiate between them for a DB upgrade
between
>>>>> versions.
>>>>>>>>>> 
>>>>>>>>>> -Chris
>>>>>>>>>> --
>>>>>>>>>> Chris Suich
>>>>>>>>>> chris.suich@netapp.com
>>>>>>>>>> NetApp Software Engineer
>>>>>>>>>> Data Center Platforms – Cloud Solutions
>>>>>>>>>> Citrix, Cisco & Red Hat
>>>>>>>>>> 
>>>>>>>>>> On Jan 24, 2014, at 9:53 AM, Mike Tutkowski <
>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>>> 
>>>>>>>>>>> I think at some point we need to use a key/value
for storage tags
>>>>> such
>>>>>>>>>> as
>>>>>>>>>>> the following:
>>>>>>>>>>> 
>>>>>>>>>>> storageTags=value1,value2,value3
>>>>>>>>>>> 
>>>>>>>>>>> The problem with that is you have to parse the
value cell each
>>> time
>>>>> you
>>>>>>>>>>> pull it out of the DB.
>>>>>>>>>>> 
>>>>>>>>>>> That might be too risky of a change, though,
for 4.3 at this
>>> point.
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> On Fri, Jan 24, 2014 at 5:24 AM, SuichII, Christopher
<
>>>>>>>>>>> Chris.Suich@netapp.com> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> I have found an additional issue related
to this. The allocators
>>> do
>>>>>>>>>>>> properly ignore any storage pool details
whose value is true that
>>>>> is
>>>>>>>>>> not
>>>>>>>>>>>> actually a storage pool. However, the list
storage pools API does
>>>>>>>>>> NOT. When
>>>>>>>>>>>> creating the StoragePoolResponse, it is still
assumed that any
>>>>>>>>>> storage pool
>>>>>>>>>>>> detail with the value ‘true’ is a storage
tag.
>>>>>>>>>>>> 
>>>>>>>>>>>> For my plugin targeting 4.3, we create a
storage pool detail
>>> with a
>>>>>>>>>>>> true/false value, so this causes a problem
with the storage pool
>>>>> UI.
>>>>>>>>>>>> 
>>>>>>>>>>>> Any thoughts on how to fix this?
>>>>>>>>>>>> 
>>>>>>>>>>>> -Chris
>>>>>>>>>>>> --
>>>>>>>>>>>> Chris Suich
>>>>>>>>>>>> chris.suich@netapp.com
>>>>>>>>>>>> NetApp Software Engineer
>>>>>>>>>>>> Data Center Platforms – Cloud Solutions
>>>>>>>>>>>> Citrix, Cisco & Red Hat
>>>>>>>>>>>> 
>>>>>>>>>>>> On Oct 23, 2013, at 6:43 PM, Alena Prokharchyk
<
>>>>>>>>>>>> Alena.Prokharchyk@citrix.com> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> Filed
>>>>>>>>>>>>> 
>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-4942
>>>>>>>>>>>>> 
>>>>>>>>>>>>> -Alena.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> From: Prachi Damle <Prachi.Damle@citrix.com<mailto:
>>>>>>>>>>>> Prachi.Damle@citrix.com>>
>>>>>>>>>>>>> Date: Wednesday, October 23, 2013 2:04
PM
>>>>>>>>>>>>> To: Alena Prokharchyk <alena.prokharchyk@citrix.com<mailto:
>>>>>>>>>>>> alena.prokharchyk@citrix.com>>, "dev@cloudstack.apache.org
>>> <mailto:
>>>>>>>>>>>> dev@cloudstack.apache.org>" <dev@cloudstack.apache.org<mailto:
>>>>>>>>>>>> dev@cloudstack.apache.org>>
>>>>>>>>>>>>> Subject: RE: Tags on storagePool
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Alena,
>>>>>>>>>>>>> 
>>>>>>>>>>>>> I don’t know why it was designed this
way in first place – a
>>>>> design
>>>>>>>>>> like
>>>>>>>>>>>> host_tags where we have separate table to
store tags is much
>>> better
>>>>>>>>>> for
>>>>>>>>>>>> Allocators to work on.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> It is a bug, but will cause problem only
if we land up with
>>>>> situation
>>>>>>>>>>>> explained below:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Given the existing design of storage
tags, the Allocators search
>>>>> the
>>>>>>>>>>>> details table using the name = <tag-provided-in-disk-offering>
>>> and
>>>>>>>>>> value
>>>>>>>>>>>> =true
>>>>>>>>>>>>> Thus this will cause a problem in placement
only if some other
>>>>>>>>>> storage
>>>>>>>>>>>> pool detail happen to have the same ‘name’
as a storage-tag and
>>>>> also a
>>>>>>>>>>>> value = true.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> -Prachi
>>>>>>>>>>>>> 
>>>>>>>>>>>>> From: Alena Prokharchyk
>>>>>>>>>>>>> Sent: Wednesday, October 23, 2013 1:36
PM
>>>>>>>>>>>>> To: Prachi Damle; dev@cloudstack.apache.org<mailto:
>>>>>>>>>>>> dev@cloudstack.apache.org>
>>>>>>>>>>>>> Subject: Tags on storagePool
>>>>>>>>>>>>> 
>>>>>>>>>>>>> I came across a potential bug in the
way allocators do volumes
>>>>>>>>>> placement
>>>>>>>>>>>> on storage, based on storage tags. Prachi,
can you please confirm
>>>>> if
>>>>>>>>>> the
>>>>>>>>>>>> bug is real.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> The tags are passed to the createStoragePool
API in form of
>>>>>>>>>>>> tags='tag1,tag2,..':
>>>>>>>>>>>>> 
>>>>>>>>>>>>> ?command=createStoragePool&...&tags=alena
>>>>>>>>>>>>> 
>>>>>>>>>>>>> and stored in the storage_pool_details
db table as:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> mysql> select *from storage_pool_details
where pool_id=2;
>>>>>>>>>>>>> +----+---------+-----------+-------+
>>>>>>>>>>>>> | id | pool_id | name      | value |
>>>>>>>>>>>>> +----+---------+-----------+-------+
>>>>>>>>>>>>> |  2 |       2 | alenatags | true  |
>>>>>>>>>>>>> +----+---------+-----------+-------+
>>>>>>>>>>>>> 1 row in set (0.00 sec)
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Allocator code assumes that everything
stored in
>>>>> storage_pool_details
>>>>>>>>>>>> table, having value=true - is a storage pool
tag. And this is
>>>>>>>>>> incorrect, as
>>>>>>>>>>>> the storage_pool_details table is used for
storing diff kinds of
>>>>>>>>>> storage
>>>>>>>>>>>> pool details - not just tags - that can be
later used by various
>>>>>>>>>> cloudStack
>>>>>>>>>>>> managers. Like for example we save Storage
level config
>>> parameters
>>>>> in
>>>>>>>>>> this
>>>>>>>>>>>> table (
>>>>>>>>>>>> 
>>>>>>>>>> 
>>>>> 
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/FS+for+Granular+Global+Configuration+Parameters
>>>>>>>>>>>> ).
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> The correct way to fix it would be -
store tags as:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> +----+---------+-----------+-------+
>>>>>>>>>>>>> | id | pool_id | name      | value |
>>>>>>>>>>>>> +----+---------+-----------+-------+
>>>>>>>>>>>>> |  2 |       2 | tag       | alena  |
>>>>>>>>>>>>> +----+---------+-----------+-------+
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> and fix StorageManager to retrive all
the tags by the "tag" key.
>>>>> We
>>>>>>>>>> also
>>>>>>>>>>>> have to fix the DB upgrade, which can be
tricky as we will have
>>> to
>>>>>>>>>> figure
>>>>>>>>>>>> out which detail is a tag.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> -Alena.
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> --
>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>> *™*
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> --
>>>>>>>>> *Mike Tutkowski*
>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>> o: 303.746.7302
>>>>>>>>> Advancing the way the world uses the cloud<
>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>> *™*
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> --
>>>>>>>> *Mike Tutkowski*
>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>> o: 303.746.7302
>>>>>>>> Advancing the way the world uses the cloud<
>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>> *™*
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> --
>>>>>>> *Mike Tutkowski*
>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>> o: 303.746.7302
>>>>>>> Advancing the way the world uses the
>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>> *™*
>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>>> 
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkowski@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the
>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>> *™*
>>> 
>>> 
>> 
>> 
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the
>> cloud<http://solidfire.com/solution/overview/?video=play>
>> *™*


Mime
View raw message