cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Prachi Damle <Prachi.Da...@citrix.com>
Subject RE: Tags on storagePool
Date Mon, 27 Jan 2014 22:23:47 GMT
There is already a ticket logged for this, https://issues.apache.org/jira/browse/CLOUDSTACK-4942

Fix should be introducing a new table say storage_tag and follow the design of host_tag and
also upgrade steps for existing storage tags present in the details.

Prachi


-----Original Message-----
From: Marcus Sorensen [mailto:shadowsor@gmail.com] 
Sent: Monday, January 27, 2014 11:32 AM
To: dev@cloudstack.apache.org
Subject: Re: Tags on storagePool

Right, for some reason I was thinking in the moment that all details would have a value of
'true'. Since that's not the case, and this field is clearly not a boolean field, we should
just change them to storage_tag for a value, as suggested.

On Mon, Jan 27, 2014 at 12:28 PM, Mike Tutkowski <mike.tutkowski@solidfire.com> wrote:
> I agree...we should fix it in 4.4.
>
> Would you write up a JIRA ticket for it, Chris?
>
> Thanks!
>
>
> On Mon, Jan 27, 2014 at 12:26 PM, SuichII, Christopher < 
> Chris.Suich@netapp.com> wrote:
>
>> I suspect the reason you don't have this problem is that you don't 
>> have any storage pool details whose value is 'true'. That is the only 
>> case where this problem arises.
>>
>> I think we do have a problem differentiating, though. If you list all 
>> of the storage tags for a storage pool, then any storage pool details 
>> whose value is 'true' will be assumed to be a storage tag. The only 
>> way around this problem I see is to cross reference the results there 
>> with all of the storage tags from disk offerings, and if you find a 
>> storage pool detail that doesn't match a disk offering storage tag, 
>> then assume it isn't a storage tag.
>>
>> Unfortunately it sounds like this is way out of scope for 4.3. I 
>> think we need to start thinking about how we can fix this issue 
>> moving forward now that there will be plugins out there impacted by this.
>>
>> -Chris
>> --
>> Chris Suich
>> chris.suich@netapp.com
>> NetApp Software Engineer
>> Data Center Platforms - Cloud Solutions Citrix, Cisco & Red Hat
>>
>> On Jan 27, 2014, at 2:21 PM, Marcus Sorensen <shadowsor@gmail.com> wrote:
>>
>> > I'm ok with whatever. It does seem like we have the ability to 
>> > differentiate real storage tags from details, since we can 
>> > list/edit those storage tags (I don't see all of my details showing 
>> > up in the UI when I look at the tags for my primary storage), 
>> > though I'm not immediately sure why or how at the moment. I think 
>> > this affects Mike the most, since he's shipping a plugin to 
>> > customers. I can just look at whatever changes are made and update my pools
manually, if need be.
>> >
>> > On Mon, Jan 27, 2014 at 11:43 AM, Mike Tutkowski 
>> > <mike.tutkowski@solidfire.com> wrote:
>> >> I agree...it certainly is not an ideal situation.
>> >>
>> >> Have you assessed the risk involved with changing storage tags 
>> >> from
>> using
>> >> true as a value to using something like storage_tag as a value?
>> >>
>> >> If so, do you feel it is an acceptable amount of risk for 4.3 now 
>> >> that
>> we
>> >> have already begun spinning up RC builds?
>> >>
>> >> Thanks
>> >>
>> >>
>> >> On Mon, Jan 27, 2014 at 8:40 AM, SuichII, Christopher < 
>> >> Chris.Suich@netapp.com> wrote:
>> >>
>> >>> I think my only real concern with working around it in this 
>> >>> manner is
>> that
>> >>> once a fix is applied, we now have a bunch of settings that do 
>> >>> not
>> conform
>> >>> to the true/false pattern. In order to migrate them from using 
>> >>> t/f,
>> yes/no
>> >>> or enabled/disabled back to using true/false, we would have to 
>> >>> write
>> some
>> >>> kind of DB upgrade/migration in our plugin to update these 
>> >>> values. In addition, I would have to change the ConfigKey from 
>> >>> being a boolean to
>> a
>> >>> string and then back to a boolean once a solution is implemented. 
>> >>> It is just some technical debt that would pile up for plugins 
>> >>> with a pretty
>> nasty
>> >>> fix down the road.
>> >>>
>> >>> -Chris
>> >>> --
>> >>> Chris Suich
>> >>> chris.suich@netapp.com
>> >>> NetApp Software Engineer
>> >>> Data Center Platforms - Cloud Solutions Citrix, Cisco & Red Hat
>> >>>
>> >>> On Jan 27, 2014, at 10:33 AM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com>
>> >>> wrote:
>> >>>
>> >>>> It is a problem, but I think - at the moment - only NetApp, 
>> >>>> SolidFire,
>> >>> and
>> >>>> Marcus have storage plug-ins.
>> >>>>
>> >>>> We could document this issue and the easy workaround of not 
>> >>>> using true
>> >>> and
>> >>>> false as a value and try to address it in 4.4.
>> >>>>
>> >>>>
>> >>>> On Mon, Jan 27, 2014 at 7:53 AM, SuichII, Christopher < 
>> >>>> Chris.Suich@netapp.com> wrote:
>> >>>>
>> >>>>> Any final thoughts on this? Letting this go into 4.3 could
>> potentially
>> >>>>> cause issues with anyone using the configuration system for

>> >>>>> plugins
>> in
>> >>> 4.3.
>> >>>>>
>> >>>>> -Chris
>> >>>>> --
>> >>>>> Chris Suich
>> >>>>> chris.suich@netapp.com
>> >>>>> NetApp Software Engineer
>> >>>>> Data Center Platforms - Cloud Solutions Citrix, Cisco &
Red Hat
>> >>>>>
>> >>>>> On Jan 24, 2014, at 3:34 PM, SuichII, Christopher <
>> >>> Chris.Suich@netapp.com>
>> >>>>> wrote:
>> >>>>>
>> >>>>>> This is the approach we'll have to take if we can't fix
this 
>> >>>>>> in
>> 4.3. It
>> >>>>> certainly works, but it isn't ideal.
>> >>>>>>
>> >>>>>> Does anyone else have any thoughts on the impact this might

>> >>>>>> have to
>> >>> 4.3?
>> >>>>>>
>> >>>>>> -Chris
>> >>>>>> --
>> >>>>>> Chris Suich
>> >>>>>> chris.suich@netapp.com
>> >>>>>> NetApp Software Engineer
>> >>>>>> Data Center Platforms - Cloud Solutions Citrix, Cisco &
Red 
>> >>>>>> Hat
>> >>>>>>
>> >>>>>> On Jan 24, 2014, at 10:11 AM, Mike Tutkowski <
>> >>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>
>> >>>>>>> This isn't a great solution, but you could also change
the 
>> >>>>>>> value
>> for
>> >>>>> your
>> >>>>>>> plug-in from true or false to something like t or f.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On Fri, Jan 24, 2014 at 8:08 AM, Mike Tutkowski <

>> >>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>
>> >>>>>>>> Edison could confirm this perhaps, but I doubt any
current
>> >>> installation
>> >>>>>>>> would have true for the value unless it was for
a storage 
>> >>>>>>>> tag (the
>> >>>>> plug-in
>> >>>>>>>> framework just came out in 4.2).
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> On Fri, Jan 24, 2014 at 8:05 AM, Mike Tutkowski
< 
>> >>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>
>> >>>>>>>>> I think your idea would be acceptable risk for
4.3. The 
>> >>>>>>>>> upgrade
>> >>> logic
>> >>>>>>>>> would have to perform this true-to-storage_tag
conversion, 
>> >>>>>>>>> too,
>> >>>>> though.
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> On Fri, Jan 24, 2014 at 8:00 AM, SuichII, Christopher
< 
>> >>>>>>>>> Chris.Suich@netapp.com> wrote:
>> >>>>>>>>>
>> >>>>>>>>>> I think the quickest, easiest change would
be to keep 
>> >>>>>>>>>> using the
>> tag
>> >>>>> name
>> >>>>>>>>>> as the key in the details table, but use
a unique value 
>> >>>>>>>>>> like
>> >>>>> 'storage_tag'
>> >>>>>>>>>> instead of 'true'. This wouldn't require
any major logic
>> changes,
>> >>>>> just the
>> >>>>>>>>>> value used to check whether something is
a storage tag.
>> >>>>>>>>>>
>> >>>>>>>>>> It is quite risky for 4.3. However my concern
is that if 
>> >>>>>>>>>> we let
>> >>> this
>> >>>>>>>>>> ship with 4.3, then any plugin that wants
to use a storage 
>> >>>>>>>>>> pool
>> >>>>> detail with
>> >>>>>>>>>> the value 'true' will make updating the
storage tag system 
>> >>>>>>>>>> MUCH
>> >>> more
>> >>>>>>>>>> difficult. As far as I can tell, there is
no other way to
>> determine
>> >>>>> if
>> >>>>>>>>>> something is a storage tag than checking
if the details 
>> >>>>>>>>>> table
>> value
>> >>>>> is
>> >>>>>>>>>> 'true'. If there are other details with
the value 'true', 
>> >>>>>>>>>> then
>> we
>> >>>>> wouldn't
>> >>>>>>>>>> be able to differentiate between them for
a DB upgrade 
>> >>>>>>>>>> between
>> >>>>> versions.
>> >>>>>>>>>>
>> >>>>>>>>>> -Chris
>> >>>>>>>>>> --
>> >>>>>>>>>> Chris Suich
>> >>>>>>>>>> chris.suich@netapp.com
>> >>>>>>>>>> NetApp Software Engineer
>> >>>>>>>>>> Data Center Platforms - Cloud Solutions
Citrix, Cisco & 
>> >>>>>>>>>> Red Hat
>> >>>>>>>>>>
>> >>>>>>>>>> On Jan 24, 2014, at 9:53 AM, Mike Tutkowski
< 
>> >>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>>> I think at some point we need to use
a key/value for 
>> >>>>>>>>>>> storage
>> tags
>> >>>>> such
>> >>>>>>>>>> as
>> >>>>>>>>>>> the following:
>> >>>>>>>>>>>
>> >>>>>>>>>>> storageTags=value1,value2,value3
>> >>>>>>>>>>>
>> >>>>>>>>>>> The problem with that is you have to
parse the value cell 
>> >>>>>>>>>>> each
>> >>> time
>> >>>>> you
>> >>>>>>>>>>> pull it out of the DB.
>> >>>>>>>>>>>
>> >>>>>>>>>>> That might be too risky of a change,
though, for 4.3 at 
>> >>>>>>>>>>> this
>> >>> point.
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> On Fri, Jan 24, 2014 at 5:24 AM, SuichII,
Christopher < 
>> >>>>>>>>>>> Chris.Suich@netapp.com> wrote:
>> >>>>>>>>>>>
>> >>>>>>>>>>>> I have found an additional issue
related to this. The
>> allocators
>> >>> do
>> >>>>>>>>>>>> properly ignore any storage pool
details whose value is 
>> >>>>>>>>>>>> true
>> that
>> >>>>> is
>> >>>>>>>>>> not
>> >>>>>>>>>>>> actually a storage pool. However,
the list storage pools 
>> >>>>>>>>>>>> API
>> does
>> >>>>>>>>>> NOT. When
>> >>>>>>>>>>>> creating the StoragePoolResponse,
it is still assumed 
>> >>>>>>>>>>>> that any
>> >>>>>>>>>> storage pool
>> >>>>>>>>>>>> detail with the value 'true' is
a storage tag.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> For my plugin targeting 4.3, we
create a storage pool 
>> >>>>>>>>>>>> detail
>> >>> with a
>> >>>>>>>>>>>> true/false value, so this causes
a problem with the 
>> >>>>>>>>>>>> storage
>> pool
>> >>>>> UI.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Any thoughts on how to fix this?
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> -Chris
>> >>>>>>>>>>>> --
>> >>>>>>>>>>>> Chris Suich
>> >>>>>>>>>>>> chris.suich@netapp.com
>> >>>>>>>>>>>> NetApp Software Engineer Data Center
Platforms - Cloud 
>> >>>>>>>>>>>> Solutions Citrix, Cisco & Red
Hat
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> On Oct 23, 2013, at 6:43 PM, Alena
Prokharchyk < 
>> >>>>>>>>>>>> Alena.Prokharchyk@citrix.com>
wrote:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>> Filed
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-4942
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> -Alena.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> From: Prachi Damle <Prachi.Damle@citrix.com<mailto:
>> >>>>>>>>>>>> Prachi.Damle@citrix.com>>
>> >>>>>>>>>>>>> Date: Wednesday, October 23,
2013 2:04 PM
>> >>>>>>>>>>>>> To: Alena Prokharchyk <alena.prokharchyk@citrix.com<mailto:
>> >>>>>>>>>>>> alena.prokharchyk@citrix.com>>,

>> >>>>>>>>>>>> "dev@cloudstack.apache.org
>> >>> <mailto:
>> >>>>>>>>>>>> dev@cloudstack.apache.org>" <dev@cloudstack.apache.org
>> <mailto:
>> >>>>>>>>>>>> dev@cloudstack.apache.org>>
>> >>>>>>>>>>>>> Subject: RE: Tags on storagePool
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Alena,
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> I don't know why it was designed
this way in first 
>> >>>>>>>>>>>>> place - a
>> >>>>> design
>> >>>>>>>>>> like
>> >>>>>>>>>>>> host_tags where we have separate
table to store tags is 
>> >>>>>>>>>>>> much
>> >>> better
>> >>>>>>>>>> for
>> >>>>>>>>>>>> Allocators to work on.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> It is a bug, but will cause
problem only if we land up 
>> >>>>>>>>>>>>> with
>> >>>>> situation
>> >>>>>>>>>>>> explained below:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Given the existing design of
storage tags, the 
>> >>>>>>>>>>>>> Allocators
>> search
>> >>>>> the
>> >>>>>>>>>>>> details table using the name = 
>> >>>>>>>>>>>> <tag-provided-in-disk-offering>
>> >>> and
>> >>>>>>>>>> value
>> >>>>>>>>>>>> =true
>> >>>>>>>>>>>>> Thus this will cause a problem
in placement only if 
>> >>>>>>>>>>>>> some
>> other
>> >>>>>>>>>> storage
>> >>>>>>>>>>>> pool detail happen to have the same
'name' as a 
>> >>>>>>>>>>>> storage-tag
>> and
>> >>>>> also a
>> >>>>>>>>>>>> value = true.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> -Prachi
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> From: Alena Prokharchyk
>> >>>>>>>>>>>>> Sent: Wednesday, October 23,
2013 1:36 PM
>> >>>>>>>>>>>>> To: Prachi Damle; dev@cloudstack.apache.org<mailto:
>> >>>>>>>>>>>> dev@cloudstack.apache.org>
>> >>>>>>>>>>>>> Subject: Tags on storagePool
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> I came across a potential bug
in the way allocators do
>> volumes
>> >>>>>>>>>> placement
>> >>>>>>>>>>>> on storage, based on storage tags.
Prachi, can you 
>> >>>>>>>>>>>> please
>> confirm
>> >>>>> if
>> >>>>>>>>>> the
>> >>>>>>>>>>>> bug is real.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> The tags are passed to the createStoragePool
API in 
>> >>>>>>>>>>>>> form of
>> >>>>>>>>>>>> tags='tag1,tag2,..':
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> ?command=createStoragePool&...&tags=alena
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> and stored in the storage_pool_details
db table as:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> mysql> select *from storage_pool_details
where 
>> >>>>>>>>>>>>> mysql> pool_id=2;
>> >>>>>>>>>>>>> +----+---------+-----------+-------+
>> >>>>>>>>>>>>> | id | pool_id | name      |
value |
>> >>>>>>>>>>>>> +----+---------+-----------+-------+
>> >>>>>>>>>>>>> |  2 |       2 | alenatags |
true  |
>> >>>>>>>>>>>>> +----+---------+-----------+-------+
>> >>>>>>>>>>>>> 1 row in set (0.00 sec)
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Allocator code assumes that
everything stored in
>> >>>>> storage_pool_details
>> >>>>>>>>>>>> table, having value=true - is a
storage pool tag. And 
>> >>>>>>>>>>>> this is
>> >>>>>>>>>> incorrect, as
>> >>>>>>>>>>>> the storage_pool_details table is
used for storing diff 
>> >>>>>>>>>>>> kinds
>> of
>> >>>>>>>>>> storage
>> >>>>>>>>>>>> pool details - not just tags - that
can be later used by
>> various
>> >>>>>>>>>> cloudStack
>> >>>>>>>>>>>> managers. Like for example we save
Storage level config
>> >>> parameters
>> >>>>> in
>> >>>>>>>>>> this
>> >>>>>>>>>>>> table (
>> >>>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>
>> >>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/FS+for+Granula
>> r+Global+Configuration+Parameters
>> >>>>>>>>>>>> ).
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> The correct way to fix it would
be - store tags as:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> +----+---------+-----------+-------+
>> >>>>>>>>>>>>> | id | pool_id | name      |
value |
>> >>>>>>>>>>>>> +----+---------+-----------+-------+
>> >>>>>>>>>>>>> |  2 |       2 | tag       |
alena  |
>> >>>>>>>>>>>>> +----+---------+-----------+-------+
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> and fix StorageManager to retrive
all the tags by the "tag"
>> key.
>> >>>>> We
>> >>>>>>>>>> also
>> >>>>>>>>>>>> have to fix the DB upgrade, which
can be tricky as we 
>> >>>>>>>>>>>> will
>> have
>> >>> to
>> >>>>>>>>>> figure
>> >>>>>>>>>>>> out which detail is a tag.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> -Alena.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> --
>> >>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire
Inc.*
>> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>> Advancing the way the world uses the

>> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>> *(tm)*
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> --
>> >>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>> o: 303.746.7302
>> >>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>> *(tm)*
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> --
>> >>>>>>>> *Mike Tutkowski*
>> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>> o: 303.746.7302
>> >>>>>>>> Advancing the way the world uses the cloud<
>> >>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>> *(tm)*
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> --
>> >>>>>>> *Mike Tutkowski*
>> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>> o: 303.746.7302
>> >>>>>>> Advancing the way the world uses the 
>> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>> *(tm)*
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> *Mike Tutkowski*
>> >>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>> e: mike.tutkowski@solidfire.com
>> >>>> o: 303.746.7302
>> >>>> Advancing the way the world uses the 
>> >>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>> *(tm)*
>> >>>
>> >>>
>> >>
>> >>
>> >> --
>> >> *Mike Tutkowski*
>> >> *Senior CloudStack Developer, SolidFire Inc.*
>> >> e: mike.tutkowski@solidfire.com
>> >> o: 303.746.7302
>> >> Advancing the way the world uses the 
>> >> cloud<http://solidfire.com/solution/overview/?video=play>
>> >> *(tm)*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *(tm)*

Mime
View raw message