cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcus Sorensen <shadow...@gmail.com>
Subject Re: Tags on storagePool
Date Mon, 27 Jan 2014 19:32:08 GMT
Right, for some reason I was thinking in the moment that all details
would have a value of 'true'. Since that's not the case, and this
field is clearly not a boolean field, we should just change them to
storage_tag for a value, as suggested.

On Mon, Jan 27, 2014 at 12:28 PM, Mike Tutkowski
<mike.tutkowski@solidfire.com> wrote:
> I agree...we should fix it in 4.4.
>
> Would you write up a JIRA ticket for it, Chris?
>
> Thanks!
>
>
> On Mon, Jan 27, 2014 at 12:26 PM, SuichII, Christopher <
> Chris.Suich@netapp.com> wrote:
>
>> I suspect the reason you don’t have this problem is that you don’t have
>> any storage pool details whose value is ‘true’. That is the only case where
>> this problem arises.
>>
>> I think we do have a problem differentiating, though. If you list all of
>> the storage tags for a storage pool, then any storage pool details whose
>> value is ‘true’ will be assumed to be a storage tag. The only way around
>> this problem I see is to cross reference the results there with all of the
>> storage tags from disk offerings, and if you find a storage pool detail
>> that doesn’t match a disk offering storage tag, then assume it isn’t a
>> storage tag.
>>
>> Unfortunately it sounds like this is way out of scope for 4.3. I think we
>> need to start thinking about how we can fix this issue moving forward now
>> that there will be plugins out there impacted by this.
>>
>> -Chris
>> --
>> Chris Suich
>> chris.suich@netapp.com
>> NetApp Software Engineer
>> Data Center Platforms – Cloud Solutions
>> Citrix, Cisco & Red Hat
>>
>> On Jan 27, 2014, at 2:21 PM, Marcus Sorensen <shadowsor@gmail.com> wrote:
>>
>> > I'm ok with whatever. It does seem like we have the ability to
>> > differentiate real storage tags from details, since we can list/edit
>> > those storage tags (I don't see all of my details showing up in the UI
>> > when I look at the tags for my primary storage), though I'm not
>> > immediately sure why or how at the moment. I think this affects Mike
>> > the most, since he's shipping a plugin to customers. I can just look
>> > at whatever changes are made and update my pools manually, if need be.
>> >
>> > On Mon, Jan 27, 2014 at 11:43 AM, Mike Tutkowski
>> > <mike.tutkowski@solidfire.com> wrote:
>> >> I agree...it certainly is not an ideal situation.
>> >>
>> >> Have you assessed the risk involved with changing storage tags from
>> using
>> >> true as a value to using something like storage_tag as a value?
>> >>
>> >> If so, do you feel it is an acceptable amount of risk for 4.3 now that
>> we
>> >> have already begun spinning up RC builds?
>> >>
>> >> Thanks
>> >>
>> >>
>> >> On Mon, Jan 27, 2014 at 8:40 AM, SuichII, Christopher <
>> >> Chris.Suich@netapp.com> wrote:
>> >>
>> >>> I think my only real concern with working around it in this manner is
>> that
>> >>> once a fix is applied, we now have a bunch of settings that do not
>> conform
>> >>> to the true/false pattern. In order to migrate them from using t/f,
>> yes/no
>> >>> or enabled/disabled back to using true/false, we would have to write
>> some
>> >>> kind of DB upgrade/migration in our plugin to update these values. In
>> >>> addition, I would have to change the ConfigKey from being a boolean
to
>> a
>> >>> string and then back to a boolean once a solution is implemented. It
is
>> >>> just some technical debt that would pile up for plugins with a pretty
>> nasty
>> >>> fix down the road.
>> >>>
>> >>> -Chris
>> >>> --
>> >>> Chris Suich
>> >>> chris.suich@netapp.com
>> >>> NetApp Software Engineer
>> >>> Data Center Platforms – Cloud Solutions
>> >>> Citrix, Cisco & Red Hat
>> >>>
>> >>> On Jan 27, 2014, at 10:33 AM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com>
>> >>> wrote:
>> >>>
>> >>>> It is a problem, but I think - at the moment - only NetApp, SolidFire,
>> >>> and
>> >>>> Marcus have storage plug-ins.
>> >>>>
>> >>>> We could document this issue and the easy workaround of not using
true
>> >>> and
>> >>>> false as a value and try to address it in 4.4.
>> >>>>
>> >>>>
>> >>>> On Mon, Jan 27, 2014 at 7:53 AM, SuichII, Christopher <
>> >>>> Chris.Suich@netapp.com> wrote:
>> >>>>
>> >>>>> Any final thoughts on this? Letting this go into 4.3 could
>> potentially
>> >>>>> cause issues with anyone using the configuration system for
plugins
>> in
>> >>> 4.3.
>> >>>>>
>> >>>>> -Chris
>> >>>>> --
>> >>>>> Chris Suich
>> >>>>> chris.suich@netapp.com
>> >>>>> NetApp Software Engineer
>> >>>>> Data Center Platforms – Cloud Solutions
>> >>>>> Citrix, Cisco & Red Hat
>> >>>>>
>> >>>>> On Jan 24, 2014, at 3:34 PM, SuichII, Christopher <
>> >>> Chris.Suich@netapp.com>
>> >>>>> wrote:
>> >>>>>
>> >>>>>> This is the approach we’ll have to take if we can’t
fix this in
>> 4.3. It
>> >>>>> certainly works, but it isn’t ideal.
>> >>>>>>
>> >>>>>> Does anyone else have any thoughts on the impact this might
have to
>> >>> 4.3?
>> >>>>>>
>> >>>>>> -Chris
>> >>>>>> --
>> >>>>>> Chris Suich
>> >>>>>> chris.suich@netapp.com
>> >>>>>> NetApp Software Engineer
>> >>>>>> Data Center Platforms – Cloud Solutions
>> >>>>>> Citrix, Cisco & Red Hat
>> >>>>>>
>> >>>>>> On Jan 24, 2014, at 10:11 AM, Mike Tutkowski <
>> >>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>
>> >>>>>>> This isn't a great solution, but you could also change
the value
>> for
>> >>>>> your
>> >>>>>>> plug-in from true or false to something like t or f.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On Fri, Jan 24, 2014 at 8:08 AM, Mike Tutkowski <
>> >>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>
>> >>>>>>>> Edison could confirm this perhaps, but I doubt any
current
>> >>> installation
>> >>>>>>>> would have true for the value unless it was for
a storage tag (the
>> >>>>> plug-in
>> >>>>>>>> framework just came out in 4.2).
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> On Fri, Jan 24, 2014 at 8:05 AM, Mike Tutkowski
<
>> >>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>
>> >>>>>>>>> I think your idea would be acceptable risk for
4.3. The upgrade
>> >>> logic
>> >>>>>>>>> would have to perform this true-to-storage_tag
conversion, too,
>> >>>>> though.
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> On Fri, Jan 24, 2014 at 8:00 AM, SuichII, Christopher
<
>> >>>>>>>>> Chris.Suich@netapp.com> wrote:
>> >>>>>>>>>
>> >>>>>>>>>> I think the quickest, easiest change would
be to keep using the
>> tag
>> >>>>> name
>> >>>>>>>>>> as the key in the details table, but use
a unique value like
>> >>>>> ‘storage_tag’
>> >>>>>>>>>> instead of ‘true’. This wouldn’t require
any major logic
>> changes,
>> >>>>> just the
>> >>>>>>>>>> value used to check whether something is
a storage tag.
>> >>>>>>>>>>
>> >>>>>>>>>> It is quite risky for 4.3. However my concern
is that if we let
>> >>> this
>> >>>>>>>>>> ship with 4.3, then any plugin that wants
to use a storage pool
>> >>>>> detail with
>> >>>>>>>>>> the value ‘true’ will make updating
the storage tag system MUCH
>> >>> more
>> >>>>>>>>>> difficult. As far as I can tell, there is
no other way to
>> determine
>> >>>>> if
>> >>>>>>>>>> something is a storage tag than checking
if the details table
>> value
>> >>>>> is
>> >>>>>>>>>> ‘true’. If there are other details with
the value ‘true’, then
>> we
>> >>>>> wouldn’t
>> >>>>>>>>>> be able to differentiate between them for
a DB upgrade between
>> >>>>> versions.
>> >>>>>>>>>>
>> >>>>>>>>>> -Chris
>> >>>>>>>>>> --
>> >>>>>>>>>> Chris Suich
>> >>>>>>>>>> chris.suich@netapp.com
>> >>>>>>>>>> NetApp Software Engineer
>> >>>>>>>>>> Data Center Platforms – Cloud Solutions
>> >>>>>>>>>> Citrix, Cisco & Red Hat
>> >>>>>>>>>>
>> >>>>>>>>>> On Jan 24, 2014, at 9:53 AM, Mike Tutkowski
<
>> >>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>>> I think at some point we need to use
a key/value for storage
>> tags
>> >>>>> such
>> >>>>>>>>>> as
>> >>>>>>>>>>> the following:
>> >>>>>>>>>>>
>> >>>>>>>>>>> storageTags=value1,value2,value3
>> >>>>>>>>>>>
>> >>>>>>>>>>> The problem with that is you have to
parse the value cell each
>> >>> time
>> >>>>> you
>> >>>>>>>>>>> pull it out of the DB.
>> >>>>>>>>>>>
>> >>>>>>>>>>> That might be too risky of a change,
though, for 4.3 at this
>> >>> point.
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> On Fri, Jan 24, 2014 at 5:24 AM, SuichII,
Christopher <
>> >>>>>>>>>>> Chris.Suich@netapp.com> wrote:
>> >>>>>>>>>>>
>> >>>>>>>>>>>> I have found an additional issue
related to this. The
>> allocators
>> >>> do
>> >>>>>>>>>>>> properly ignore any storage pool
details whose value is true
>> that
>> >>>>> is
>> >>>>>>>>>> not
>> >>>>>>>>>>>> actually a storage pool. However,
the list storage pools API
>> does
>> >>>>>>>>>> NOT. When
>> >>>>>>>>>>>> creating the StoragePoolResponse,
it is still assumed that any
>> >>>>>>>>>> storage pool
>> >>>>>>>>>>>> detail with the value ‘true’
is a storage tag.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> For my plugin targeting 4.3, we
create a storage pool detail
>> >>> with a
>> >>>>>>>>>>>> true/false value, so this causes
a problem with the storage
>> pool
>> >>>>> UI.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Any thoughts on how to fix this?
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> -Chris
>> >>>>>>>>>>>> --
>> >>>>>>>>>>>> Chris Suich
>> >>>>>>>>>>>> chris.suich@netapp.com
>> >>>>>>>>>>>> NetApp Software Engineer
>> >>>>>>>>>>>> Data Center Platforms – Cloud
Solutions
>> >>>>>>>>>>>> Citrix, Cisco & Red Hat
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> On Oct 23, 2013, at 6:43 PM, Alena
Prokharchyk <
>> >>>>>>>>>>>> Alena.Prokharchyk@citrix.com>
wrote:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>> Filed
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-4942
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> -Alena.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> From: Prachi Damle <Prachi.Damle@citrix.com<mailto:
>> >>>>>>>>>>>> Prachi.Damle@citrix.com>>
>> >>>>>>>>>>>>> Date: Wednesday, October 23,
2013 2:04 PM
>> >>>>>>>>>>>>> To: Alena Prokharchyk <alena.prokharchyk@citrix.com<mailto:
>> >>>>>>>>>>>> alena.prokharchyk@citrix.com>>,
"dev@cloudstack.apache.org
>> >>> <mailto:
>> >>>>>>>>>>>> dev@cloudstack.apache.org>" <dev@cloudstack.apache.org
>> <mailto:
>> >>>>>>>>>>>> dev@cloudstack.apache.org>>
>> >>>>>>>>>>>>> Subject: RE: Tags on storagePool
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Alena,
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> I don’t know why it was designed
this way in first place – a
>> >>>>> design
>> >>>>>>>>>> like
>> >>>>>>>>>>>> host_tags where we have separate
table to store tags is much
>> >>> better
>> >>>>>>>>>> for
>> >>>>>>>>>>>> Allocators to work on.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> It is a bug, but will cause
problem only if we land up with
>> >>>>> situation
>> >>>>>>>>>>>> explained below:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Given the existing design of
storage tags, the Allocators
>> search
>> >>>>> the
>> >>>>>>>>>>>> details table using the name = <tag-provided-in-disk-offering>
>> >>> and
>> >>>>>>>>>> value
>> >>>>>>>>>>>> =true
>> >>>>>>>>>>>>> Thus this will cause a problem
in placement only if some
>> other
>> >>>>>>>>>> storage
>> >>>>>>>>>>>> pool detail happen to have the same
‘name’ as a storage-tag
>> and
>> >>>>> also a
>> >>>>>>>>>>>> value = true.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> -Prachi
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> From: Alena Prokharchyk
>> >>>>>>>>>>>>> Sent: Wednesday, October 23,
2013 1:36 PM
>> >>>>>>>>>>>>> To: Prachi Damle; dev@cloudstack.apache.org<mailto:
>> >>>>>>>>>>>> dev@cloudstack.apache.org>
>> >>>>>>>>>>>>> Subject: Tags on storagePool
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> I came across a potential bug
in the way allocators do
>> volumes
>> >>>>>>>>>> placement
>> >>>>>>>>>>>> on storage, based on storage tags.
Prachi, can you please
>> confirm
>> >>>>> if
>> >>>>>>>>>> the
>> >>>>>>>>>>>> bug is real.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> The tags are passed to the createStoragePool
API in form of
>> >>>>>>>>>>>> tags='tag1,tag2,..':
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> ?command=createStoragePool&...&tags=alena
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> and stored in the storage_pool_details
db table as:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> mysql> select *from storage_pool_details
where pool_id=2;
>> >>>>>>>>>>>>> +----+---------+-----------+-------+
>> >>>>>>>>>>>>> | id | pool_id | name      |
value |
>> >>>>>>>>>>>>> +----+---------+-----------+-------+
>> >>>>>>>>>>>>> |  2 |       2 | alenatags |
true  |
>> >>>>>>>>>>>>> +----+---------+-----------+-------+
>> >>>>>>>>>>>>> 1 row in set (0.00 sec)
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Allocator code assumes that
everything stored in
>> >>>>> storage_pool_details
>> >>>>>>>>>>>> table, having value=true - is a
storage pool tag. And this is
>> >>>>>>>>>> incorrect, as
>> >>>>>>>>>>>> the storage_pool_details table is
used for storing diff kinds
>> of
>> >>>>>>>>>> storage
>> >>>>>>>>>>>> pool details - not just tags - that
can be later used by
>> various
>> >>>>>>>>>> cloudStack
>> >>>>>>>>>>>> managers. Like for example we save
Storage level config
>> >>> parameters
>> >>>>> in
>> >>>>>>>>>> this
>> >>>>>>>>>>>> table (
>> >>>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>
>> >>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/FS+for+Granular+Global+Configuration+Parameters
>> >>>>>>>>>>>> ).
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> The correct way to fix it would
be - store tags as:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> +----+---------+-----------+-------+
>> >>>>>>>>>>>>> | id | pool_id | name      |
value |
>> >>>>>>>>>>>>> +----+---------+-----------+-------+
>> >>>>>>>>>>>>> |  2 |       2 | tag       |
alena  |
>> >>>>>>>>>>>>> +----+---------+-----------+-------+
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> and fix StorageManager to retrive
all the tags by the "tag"
>> key.
>> >>>>> We
>> >>>>>>>>>> also
>> >>>>>>>>>>>> have to fix the DB upgrade, which
can be tricky as we will
>> have
>> >>> to
>> >>>>>>>>>> figure
>> >>>>>>>>>>>> out which detail is a tag.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> -Alena.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> --
>> >>>>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire
Inc.*
>> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>>>> o: 303.746.7302
>> >>>>>>>>>>> Advancing the way the world uses the
>> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>>>> *™*
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> --
>> >>>>>>>>> *Mike Tutkowski*
>> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>>> o: 303.746.7302
>> >>>>>>>>> Advancing the way the world uses the cloud<
>> >>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>>> *™*
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> --
>> >>>>>>>> *Mike Tutkowski*
>> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>>> o: 303.746.7302
>> >>>>>>>> Advancing the way the world uses the cloud<
>> >>>>> http://solidfire.com/solution/overview/?video=play>
>> >>>>>>>> *™*
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> --
>> >>>>>>> *Mike Tutkowski*
>> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>>>>> e: mike.tutkowski@solidfire.com
>> >>>>>>> o: 303.746.7302
>> >>>>>>> Advancing the way the world uses the
>> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>>>>> *™*
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> *Mike Tutkowski*
>> >>>> *Senior CloudStack Developer, SolidFire Inc.*
>> >>>> e: mike.tutkowski@solidfire.com
>> >>>> o: 303.746.7302
>> >>>> Advancing the way the world uses the
>> >>>> cloud<http://solidfire.com/solution/overview/?video=play>
>> >>>> *™*
>> >>>
>> >>>
>> >>
>> >>
>> >> --
>> >> *Mike Tutkowski*
>> >> *Senior CloudStack Developer, SolidFire Inc.*
>> >> e: mike.tutkowski@solidfire.com
>> >> o: 303.746.7302
>> >> Advancing the way the world uses the
>> >> cloud<http://solidfire.com/solution/overview/?video=play>
>> >> *™*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *™*

Mime
View raw message