cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Tutkowski <mike.tutkow...@solidfire.com>
Subject Re: [MERGE] disk_io_throttling to MASTER
Date Fri, 14 Jun 2013 22:10:39 GMT
Right...Wei can work on the relevant code to support his feature and I'll
do mine and we can coordinate on the GUI and API mutual-exclusion logic.


On Fri, Jun 14, 2013 at 4:06 PM, John Burwell <jburwell@basho.com> wrote:

> Mike,
>
> It is important to note that we need a similar rail for throttled I/O as
> well.  Hopefully, we can work that out early next week.
>
> Thanks,
> -John
>
> On Jun 14, 2013, at 6:01 PM, Mike Tutkowski <mike.tutkowski@solidfire.com>
> wrote:
>
> > We should be on our way to getting a really good 1.0 release out for
> this,
> > so that's cool. :)
> >
> > Thanks for the time...I should be able to find time this weekend to work
> on
> > this.
> >
> > Next week I can look into the GUI and API changes for mutual exclusion of
> > mine and Wei's features.
> >
> >
> > On Fri, Jun 14, 2013 at 3:56 PM, John Burwell <jburwell@basho.com>
> wrote:
> >
> >> Mike,
> >>
> >> Awesome.  +1.
> >>
> >> Thanks for your patience with the back and forth,
> >> -John
> >>
> >> On Jun 14, 2013, at 5:50 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com>
> >> wrote:
> >>
> >>> Quick comments below in red, but I think we're good to go.
> >>>
> >>>
> >>> On Fri, Jun 14, 2013 at 3:42 PM, John Burwell <jburwell@basho.com>
> >> wrote:
> >>>
> >>>> Mike,
> >>>>
> >>>> Please see my comments in-line below.
> >>>>
> >>>> Thanks,
> >>>> -John
> >>>>
> >>>> On Jun 14, 2013, at 5:37 PM, Mike Tutkowski <
> >> mike.tutkowski@solidfire.com>
> >>>> wrote:
> >>>>
> >>>>> Comments below in red (but I believe we are reaching a consensus
> here).
> >>>> :)
> >>>>>
> >>>>>
> >>>>> On Fri, Jun 14, 2013 at 2:42 PM, John Burwell <jburwell@basho.com>
> >>>> wrote:
> >>>>>
> >>>>>> Mike,
> >>>>>>
> >>>>>> Please see my comments in-line below.
> >>>>>>
> >>>>>> Thanks,
> >>>>>> -John
> >>>>>>
> >>>>>> On Jun 14, 2013, at 4:31 PM, Mike Tutkowski <
> >>>> mike.tutkowski@solidfire.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> I am OK with that approach, John.
> >>>>>>>
> >>>>>>> So, let me review to make sure I follow you correctly:
> >>>>>>>
> >>>>>>> We introduce two new parameters to the plug-in: Number of Total
> IOPS
> >>>> for
> >>>>>>> the SAN and an overcommit ratio. (On a side note, if we are just
> >>>>>>> multiplying the two numbers, why don't we have the user just input
> >>>> their
> >>>>>>> product?)
> >>>>>>
> >>>>>> It is a straw man suggestion to allow some fine tuning of the
> >> allocation
> >>>>>> algorithm. I defer to you to determine whether or not such a tunable
> >>>> would
> >>>>>> be valuable in this use case.  My thinking was that it is tunable to
> >>>>>> support operators as they more deeply understand/observe the CS
> >> workload
> >>>>>> while allowing them to think about IOPS absolutely.  For example, I
> >>>> start
> >>>>>> out at say 50% overcommit and realize that the load is actually
> >> heavier
> >>>>>> than expected, I can reduce it or vice versa.
> >>>>>>
> >>>>>
> >>>>> Using your example of support people changing the value after seeing
> >> how
> >>>>> the system is performing:
> >>>>>
> >>>>> If we had them just pass in a new Total IOPS value, that should cover
> >> it.
> >>>>
> >>>> That is fine answer to me.  I expect that we will be refining this
> >>>> capability over the next few releases.  Therefore, I am cool with
> >> starting
> >>>> as simple as possible as it will ease/simplify future evolution.
> >>>>
> >>>>>
> >>>>>
> >>>>>>
> >>>>>>>
> >>>>>>> Are both of these new parameters to the create storage pool API
> >> command
> >>>>>> or
> >>>>>>> are they passed into the create storage pool API command through
> its
> >>>> url
> >>>>>>> parameter?
> >>>>>>>
> >>>>>>> If they are new parameters, we should make two new columns in the
> >>>>>>> storage_pool table.
> >>>>>>>
> >>>>>>> If they are passed in via the url parameter, they should go in the
> >>>>>>> storage_pool_details table.
> >>>>>>
> >>>>>> I think these allocation parameters implementation agnostic, and
> >> should
> >>>> be
> >>>>>> columns on the storage_pool table and first class properties on the
> >>>>>> StoragePool class.
> >>>>>>
> >>>>>
> >>>>> Per my most recent e-mail, I agree.
> >>>>>
> >>>>>
> >>>>>>
> >>>>>>>
> >>>>>>> For 4.2, if someone wants to change these values, they must update
> >> the
> >>>> DB
> >>>>>>> manually.
> >>>>>>
> >>>>>> Is it not possible to support updates through the updateStoragePool
> >> API
> >>>>>> call?
> >>>>>>
> >>>>>
> >>>>> Edison - if you're reading these - do you know if the plug-ins are
> >>>> informed
> >>>>> when the updateStoragePool API command is issued?
> >>>>
> >>>> The way I am thinking about these thresholds is that they are part of
> >> the
> >>>> storage engine itself.  Why would plugin need to know that the value
> >>>> changed?  When the plugin needs the value, it will be retrieved from
> the
> >>>> database ...
> >>>>
> >>>
> >>> You know, when I wrote what I wrote above, I was thinking the data was
> >>> coming in with the url parameter, which has to be parsed by the plug-in
> >>> (and it's stored in the storage_pool_details table).
> >>>
> >>> Since this is a new, standard parameter for the API (stored in the
> >>> storage_pool table), it should work just fine without plug-in
> >> interaction.
> >>>
> >>>
> >>>>
> >>>>>
> >>>>>
> >>>>>>
> >>>>>>>
> >>>>>>> Every time a volume is created or deleted for the SolidFire
> plug-in,
> >>>> the
> >>>>>>> Current IOPS value (sum of all volumes' Min IOPS that are
> associated
> >>>> with
> >>>>>>> the plug-in) is updated.
> >>>>>>
> >>>>>> I don't think we need another column for this computed value.  It
> will
> >>>> be
> >>>>>> simpler to simply ask the database to sum the values as needed.
> >>>>>>
> >>>>>
> >>>>> I'm thinking now that we should follow the pattern established by the
> >>>>> "size" field. There are two related size fields for a storage pool:
> >>>>> capacity_bytes and available bytes.
> >>>>>
> >>>>> We should just follow this pattern and create two new fields:
> >>>> capacity_iops
> >>>>> and available_iops. The plug-in can increment or decrement
> >> available_iops
> >>>>> each time a volume is created or deleted, respectively.
> >>>>
> >>>> I would rather see us do away with available_bytes.  My experience
> with
> >>>> storing computed fields is that often get out of sync and represent
> >>>> premature optimization.  I missed that part of the throttled I/O patch
> >> ...
> >>>>
> >>>
> >>> Fair enough. Agreed - one new field in the storage_pool table:
> >>> capacity_iops.
> >>>
> >>>
> >>>>
> >>>>>
> >>>>>
> >>>>>>
> >>>>>>>
> >>>>>>> The allocator can use these fields to determine if it can fit in a
> >> new
> >>>>>>> volume.
> >>>>>>
> >>>>>> Exactly.
> >>>>>>
> >>>>>>>
> >>>>>>> Does it look like my understanding is OK?
> >>>>>>
> >>>>>> Yes, looks good to me.
> >>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Fri, Jun 14, 2013 at 2:14 PM, John Burwell <jburwell@basho.com>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>>> Mike,
> >>>>>>>>
> >>>>>>>> I apologize for not being clear -- this conversation has been
> >>>> admittedly
> >>>>>>>> disjoint.  I think we should allow the maximum IOPS and
> >> overcommitment
> >>>>>>>> values to be updated though I would recommend restricting updates
> to
> >>>> to
> >>>>>> be
> >>>>>>>> an increasing value for 4.2 (e.g. users can increase the number of
> >>>> total
> >>>>>>>> IOPS from 200,000 to 250,000, but not decrease from 250,000 to
> >>>> 200,000).
> >>>>>>>> While not ideal, given the amount of time we have left for 4.2, it
> >>>> will
> >>>>>>>> cover most cases, and we can address the implications of reducing
> >>>>>> resource
> >>>>>>>> capacity in 4.3.  This approach addresses both of your concerns.
> >>>>>> First, it
> >>>>>>>> allows the administrator/operator to determine what portion of the
> >>>>>> device
> >>>>>>>> they wish to dedicate.  For example, if the device has a total
> >>>> capacity
> >>>>>> of
> >>>>>>>> 200,000 IOPS, and they only want CS to use 25% of the device then
> >> they
> >>>>>> set
> >>>>>>>> the maximum total IOPS to 50,000.  Second, as they grow capacity,
> >> they
> >>>>>> can
> >>>>>>>> update the DataStore to increase the number of IOPS they want to
> >>>>>> dedicate
> >>>>>>>> to CS' use.  I would imagine expansion of capacity happens
> >>>> infrequently
> >>>>>>>> enough that increasing the maximum IOPS value would not be a
> >>>> significant
> >>>>>>>> burden.
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> -John
> >>>>>>>>
> >>>>>>>> On Jun 14, 2013, at 4:06 PM, Mike Tutkowski <
> >>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> "the administrator/operator simply needs to tell us the total
> >> number
> >>>> of
> >>>>>>>>> IOPS that can be committed to it and an overcommitment factor."
> >>>>>>>>>
> >>>>>>>>> Are you thinking when we create a plug-in as primary storage that
> >> we
> >>>>>> say
> >>>>>>>> -
> >>>>>>>>> up front - how many IOPS the SAN can handle?
> >>>>>>>>>
> >>>>>>>>> That is not a good move, in my opinion. Our SAN is designed to
> >> start
> >>>>>>>> small
> >>>>>>>>> and grow to PBs. As the need arises for more storage, the admin
> >>>>>> purchases
> >>>>>>>>> additional storage nodes that join the cluster and the
> performance
> >>>> and
> >>>>>>>>> capacity go up.
> >>>>>>>>>
> >>>>>>>>> We need to know how many IOPS total the SAN can handle and what
> it
> >> is
> >>>>>>>>> committed to currently (the sum of the number of volumes' min
> >> IOPS).
> >>>>>>>>>
> >>>>>>>>> We also cannot assume the SAN is dedicated to CS.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Fri, Jun 14, 2013 at 1:59 PM, John Burwell <
> jburwell@basho.com>
> >>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Simon,
> >>>>>>>>>>
> >>>>>>>>>> Yes, it is CloudStack's job to protect, as best it can, from
> >>>>>>>>>> oversubscribing resources.  I would argue that resource
> management
> >>>> is
> >>>>>>>> one,
> >>>>>>>>>> if not the most, important functions of the system.  It is no
> >>>>>> different
> >>>>>>>>>> than the allocation/planning performed for hosts relative to
> cores
> >>>> and
> >>>>>>>>>> memory.  We can still oversubscribe resources, but we have
> rails +
> >>>>>> knobs
> >>>>>>>>>> and dials to avoid it.  Without these controls in place, we
> could
> >>>>>> easily
> >>>>>>>>>> allow users to deploy workloads that overrun resources harming
> all
> >>>>>>>> tenants.
> >>>>>>>>>>
> >>>>>>>>>> I also think that we are over thinking this issue for
> provisioned
> >>>>>> IOPS.
> >>>>>>>>>> When the DataStore is configured, the administrator/operator
> >> simply
> >>>>>>>> needs
> >>>>>>>>>> to tell us the total number of IOPS that can be committed to it
> >> and
> >>>> an
> >>>>>>>>>> overcommitment factor.  As we allocate volumes to that
> DataStore,
> >> we
> >>>>>>>> sum up
> >>>>>>>>>> the committed IOPS of the existing Volumes attached to the
> >>>> DataStore,
> >>>>>>>> apply
> >>>>>>>>>> the overcommitment factor, and determine whether or not the
> >>>> requested
> >>>>>>>>>> minimum IOPS for the new volume can be fulfilled.  We can
> provide
> >>>> both
> >>>>>>>>>> general and vendor specific documentation for determining these
> >>>> values
> >>>>>>>> --
> >>>>>>>>>> be they to consume the entire device or a portion of it.
> >>>>>>>>>>
> >>>>>>>>>> Querying the device is unnecessary and deceptive.  CloudStack
> >>>> resource
> >>>>>>>>>> management is not interested in the current state of the device
> >>>> which
> >>>>>>>> could
> >>>>>>>>>> be anywhere from extremely heavy to extremely light at any given
> >>>> time.
> >>>>>>>> We
> >>>>>>>>>> are interested in the worst case load that is anticipated for
> >>>>>> resource.
> >>>>>>>> In
> >>>>>>>>>> my view, it is up to administrators/operators to instrument
> their
> >>>>>>>>>> environment to understand usage patterns and capacity.  We
> should
> >>>>>>>> provide
> >>>>>>>>>> information that will help determine what should be
> >>>>>>>> instrumented/monitored,
> >>>>>>>>>> but that function should be performed outside of CloudStack.
> >>>>>>>>>>
> >>>>>>>>>> Thanks,
> >>>>>>>>>> -John
> >>>>>>>>>>
> >>>>>>>>>> On Jun 14, 2013, at 2:20 PM, Simon Weller <sweller@ena.com>
> >> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> I'd like to comment on this briefly.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> I think an assumption is being made that the SAN is being
> >> dedicated
> >>>>>> to
> >>>>>>>> a
> >>>>>>>>>> CS instance.
> >>>>>>>>>>>
> >>>>>>>>>>> My person opinion that this whole IOPS calculation is getting
> >>>> rather
> >>>>>>>>>> complicated, and could probably be much simpler than this. Over
> >>>>>>>>>> subscription is a fact of life on virtually all storage, and is
> >>>> really
> >>>>>>>> no
> >>>>>>>>>> different in concept than multiple virt instances on a single
> >> piece
> >>>> of
> >>>>>>>>>> hardware. All decent SANs offer many management options for the
> >>>>>> storage
> >>>>>>>>>> engineers to keep track of IOPS utilization, and plan for
> spindle
> >>>>>>>>>> augmentation as required.
> >>>>>>>>>>> Is it really the job of CS to become yet another management
> layer
> >>>> on
> >>>>>>>> top
> >>>>>>>>>> of this?
> >>>>>>>>>>>
> >>>>>>>>>>> ----- Original Message -----
> >>>>>>>>>>>
> >>>>>>>>>>> From: "Mike Tutkowski" <mike.tutkowski@solidfire.com>
> >>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>>> Cc: "John Burwell" <jburwell@basho.com>, "Wei Zhou" <
> >>>>>>>>>> ustcweizhou@gmail.com>
> >>>>>>>>>>> Sent: Friday, June 14, 2013 1:00:26 PM
> >>>>>>>>>>> Subject: Re: [MERGE] disk_io_throttling to MASTER
> >>>>>>>>>>>
> >>>>>>>>>>> 1) We want number of IOPS currently supported by the SAN.
> >>>>>>>>>>>
> >>>>>>>>>>> 2) We want the number of IOPS that are committed (sum of min
> IOPS
> >>>> for
> >>>>>>>>>> each
> >>>>>>>>>>> volume).
> >>>>>>>>>>>
> >>>>>>>>>>> We could do the following to keep track of IOPS:
> >>>>>>>>>>>
> >>>>>>>>>>> The plug-in could have a timer thread that goes off every,
> say, 1
> >>>>>>>> minute.
> >>>>>>>>>>>
> >>>>>>>>>>> It could query the SAN for the number of nodes that make up the
> >> SAN
> >>>>>> and
> >>>>>>>>>>> multiple this by 50,000. This is essentially the number of
> >>>> supported
> >>>>>>>> IOPS
> >>>>>>>>>>> of the SAN.
> >>>>>>>>>>>
> >>>>>>>>>>> The next API call could be to get all of the volumes on the
> SAN.
> >>>>>>>> Iterate
> >>>>>>>>>>> through them all and add up their min IOPS values. This is the
> >>>> number
> >>>>>>>> of
> >>>>>>>>>>> IOPS the SAN is committed to.
> >>>>>>>>>>>
> >>>>>>>>>>> These two numbers can then be updated in the storage_pool table
> >> (a
> >>>>>>>> column
> >>>>>>>>>>> for each value).
> >>>>>>>>>>>
> >>>>>>>>>>> The allocators can get these values as needed (and they would
> be
> >> as
> >>>>>>>>>>> accurate as the last time the thread asked the SAN for this
> >> info).
> >>>>>>>>>>>
> >>>>>>>>>>> These two fields, the min IOPS of the volume to create, and the
> >>>>>>>>>> overcommit
> >>>>>>>>>>> ratio of the plug-in would tell the allocator if it can select
> >> the
> >>>>>>>> given
> >>>>>>>>>>> storage pool.
> >>>>>>>>>>>
> >>>>>>>>>>> What do you think?
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Fri, Jun 14, 2013 at 11:45 AM, Mike Tutkowski <
> >>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> "As I mentioned previously, I am very reluctant for any
> feature
> >> to
> >>>>>>>> come
> >>>>>>>>>>>> into master that can exhaust resources."
> >>>>>>>>>>>>
> >>>>>>>>>>>> Just wanted to mention that, worst case, the SAN would fail
> >>>> creation
> >>>>>>>> of
> >>>>>>>>>>>> the volume before allowing a new volume to break the system.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Fri, Jun 14, 2013 at 11:35 AM, Mike Tutkowski <
> >>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Are you thinking we add a column on to the storage pool
> table,
> >>>>>>>>>>>>> IOPS_Count, where we add and subtract committed IOPS?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> That is easy enough.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> How do you want to determine what the SAN is capable of
> >>>> supporting
> >>>>>>>> IOPS
> >>>>>>>>>>>>> wise? Remember we're dealing with a dynamic SAN here...as you
> >> add
> >>>>>>>>>> storage
> >>>>>>>>>>>>> nodes to the cluster, the number of IOPS increases. Do we
> have
> >> a
> >>>>>>>>>> thread we
> >>>>>>>>>>>>> can use to query external devices like this SAN to update the
> >>>>>>>> supported
> >>>>>>>>>>>>> number of IOPS?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Also, how do you want to enforce the IOPS limit? Do we pass
> in
> >> an
> >>>>>>>>>>>>> overcommit ration to the plug-in when it's created? We would
> >> need
> >>>>>> to
> >>>>>>>>>> store
> >>>>>>>>>>>>> this in the storage_pool table, as well, I believe.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> We should also get Wei involved in this as his feature will
> >> need
> >>>>>>>>>> similar
> >>>>>>>>>>>>> functionality.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Also, we should do this FAST as we have only two weeks left
> and
> >>>>>> many
> >>>>>>>> of
> >>>>>>>>>>>>> us will be out for several days for the CS Collab Conference.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Fri, Jun 14, 2013 at 10:46 AM, John Burwell <
> >>>> jburwell@basho.com
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Querying the SAN only indicates the number of IOPS currently
> >> in
> >>>>>> use.
> >>>>>>>>>>>>>> The allocator needs to check the number of IOPS committed
> >> which
> >>>> is
> >>>>>>>>>> tracked
> >>>>>>>>>>>>>> by CloudStack. For 4.2, we should be able to query the
> number
> >> of
> >>>>>>>> IOPS
> >>>>>>>>>>>>>> committed to a DataStore, and determine whether or not the
> >>>> number
> >>>>>>>>>> requested
> >>>>>>>>>>>>>> can be fulfilled by that device. It seems to be that a
> >>>>>>>>>>>>>> DataStore#getCommittedIOPS() : Long method would be
> >> sufficient.
> >>>>>>>>>>>>>> DataStore's that don't support provisioned IOPS would return
> >>>> null.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> As I mentioned previously, I am very reluctant for any
> feature
> >>>> to
> >>>>>>>> come
> >>>>>>>>>>>>>> into master that can exhaust resources.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Jun 13, 2013, at 9:27 PM, Mike Tutkowski <
> >>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Yeah, I'm not sure I could come up with anything near an
> >>>> accurate
> >>>>>>>>>>>>>>> assessment of how many IOPS are currently available on the
> >> SAN
> >>>>>> (or
> >>>>>>>>>>>>>> even a
> >>>>>>>>>>>>>>> total number that are available for volumes). Not sure if
> >>>> there's
> >>>>>>>> yet
> >>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>> API call for that.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> If I did know this number (total number of IOPS supported
> by
> >>>> the
> >>>>>>>>>> SAN),
> >>>>>>>>>>>>>> we'd
> >>>>>>>>>>>>>>> still have to keep track of the total number of volumes
> we've
> >>>>>>>> created
> >>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>> CS on the SAN in terms of their IOPS. Also, if an admin
> >> issues
> >>>> an
> >>>>>>>> API
> >>>>>>>>>>>>>> call
> >>>>>>>>>>>>>>> directly to the SAN to tweak the number of IOPS on a given
> >>>> volume
> >>>>>>>> or
> >>>>>>>>>>>>>> set of
> >>>>>>>>>>>>>>> volumes (not supported from CS, but supported via the
> >> SolidFire
> >>>>>>>> API),
> >>>>>>>>>>>>>> our
> >>>>>>>>>>>>>>> numbers in CS would be off.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I'm thinking verifying sufficient number of IOPS is a
> really
> >>>> good
> >>>>>>>>>> idea
> >>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>> a future release.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> For 4.2 I think we should stick to having the allocator
> >> detect
> >>>> if
> >>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>> QoS is desired and if the storage pool in question supports
> >>>> that
> >>>>>>>>>>>>>> feature.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> If you really are over provisioned on your SAN in terms of
> >> IOPS
> >>>>>> or
> >>>>>>>>>>>>>>> capacity, the SAN can let the admin know in several
> different
> >>>>>> ways
> >>>>>>>>>>>>>> (e-mail,
> >>>>>>>>>>>>>>> SNMP, GUI).
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Thu, Jun 13, 2013 at 7:02 PM, John Burwell <
> >>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Please see my comments in-line below.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Jun 13, 2013, at 6:09 PM, Mike Tutkowski <
> >>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Comments below in red.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Thanks
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Thu, Jun 13, 2013 at 3:58 PM, John Burwell <
> >>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Overall, I agree with the steps to below for 4.2.
> However,
> >>>> we
> >>>>>>>> may
> >>>>>>>>>>>>>> want
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>> throw an exception if we can not fulfill a requested
> QoS.
> >> If
> >>>>>> the
> >>>>>>>>>>>>>> user
> >>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>> expecting that the hypervisor will provide a particular
> >> QoS,
> >>>>>> and
> >>>>>>>>>>>>>> that is
> >>>>>>>>>>>>>>>>>> not possible, it seems like we should inform them rather
> >>>> than
> >>>>>>>>>>>>>> silently
> >>>>>>>>>>>>>>>>>> ignoring their request.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Sure, that sounds reasonable.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> We'd have to come up with some way for the allocators to
> >> know
> >>>>>>>> about
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> requested storage QoS and then query the candidate
> drivers.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Any thoughts on how we might do that?
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> To collect my thoughts from previous parts of the
> thread,
> >> I
> >>>> am
> >>>>>>>>>>>>>>>>>> uncomfortable with the idea that the management server
> can
> >>>>>>>>>>>>>> overcommit a
> >>>>>>>>>>>>>>>>>> resource. You had mentioned querying the device for
> >>>> available
> >>>>>>>>>> IOPS.
> >>>>>>>>>>>>>>>> While
> >>>>>>>>>>>>>>>>>> that would be nice, it seems like we could fall back to
> a
> >>>> max
> >>>>>>>> IOPS
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>> overcommit factor manually calculated and entered by the
> >>>>>>>>>>>>>>>>>> administrator/operator. I think such threshold and
> >>>> allocation
> >>>>>>>>>> rails
> >>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>> be added for both provisioned IOPS and throttled I/O --
> it
> >>>> is
> >>>>>> a
> >>>>>>>>>>>>>> basic
> >>>>>>>>>>>>>>>>>> feature of any cloud orchestration platform.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Are you thinking this ability would make it into 4.2?
> Just
> >>>>>>>> curious
> >>>>>>>>>>>>>> what
> >>>>>>>>>>>>>>>>> release we're talking about here. For the SolidFire SAN,
> >> you
> >>>>>>>> might
> >>>>>>>>>>>>>> have,
> >>>>>>>>>>>>>>>>> say, four separate storage nodes to start (200,000 IOPS)
> >> and
> >>>>>> then
> >>>>>>>>>>>>>> later
> >>>>>>>>>>>>>>>> add
> >>>>>>>>>>>>>>>>> a new node (now you're at 250,000 IOPS). CS would have to
> >>>> have
> >>>>>> a
> >>>>>>>>>> way
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> know that the number of supported IOPS has increased.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Yes, I think we need some *basic*/conservative rails in
> 4.2.
> >>>> For
> >>>>>>>>>>>>>> example,
> >>>>>>>>>>>>>>>> we may only support expanding capacity in 4.2, and not
> >> handle
> >>>>>> any
> >>>>>>>>>>>>>>>> re-balance scenarios -- node failure, addition, etc.
> >>>>>> Extrapolating
> >>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>> bit, the throttled I/O enhancement seems like it needs a
> >>>> similar
> >>>>>>>> set
> >>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>> rails defined per host.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> For 4.3, I don't like the idea that a QoS would be
> >> expressed
> >>>>>> in
> >>>>>>>> a
> >>>>>>>>>>>>>>>>>> implementation specific manner. I think we need to
> arrive
> >>>> at a
> >>>>>>>>>>>>>> general
> >>>>>>>>>>>>>>>>>> model that can be exploited by the allocators and
> >> planners.
> >>>> We
> >>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>> restrict implementation specific key-value pairs (call
> >> them
> >>>>>>>>>> details,
> >>>>>>>>>>>>>>>>>> extended data, whatever) to information that is unique
> to
> >>>> the
> >>>>>>>>>>>>>> driver and
> >>>>>>>>>>>>>>>>>> would provide no useful information to the management
> >>>> server's
> >>>>>>>>>>>>>>>>>> orchestration functions. A resource QoS does not seem to
> >> fit
> >>>>>>>> those
> >>>>>>>>>>>>>>>>>> criteria.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I wonder if this would be a good discussion topic for
> >>>> Sunday's
> >>>>>> CS
> >>>>>>>>>>>>>> Collab
> >>>>>>>>>>>>>>>>> Conf hack day that Joe just sent out an e-mail about?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Yes, it would -- I will put something in the wiki topic.
> It
> >>>> will
> >>>>>>>>>>>>>> also be
> >>>>>>>>>>>>>>>> part of my talk on Monday -- How to Run from Zombie which
> >>>>>> include
> >>>>>>>>>>>>>> some of
> >>>>>>>>>>>>>>>> my opinions on the topic.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Jun 13, 2013, at 5:44 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> So, here's my suggestion for 4.2:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Accept the values as they are currently required (four
> >> new
> >>>>>>>> fields
> >>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>> Wei's
> >>>>>>>>>>>>>>>>>>> feature or two new fields for mine).
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> The Add Disk Offering dialog needs three new radio
> >> buttons:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> 1) No QoS
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> 2) Hypervisor QoS
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> 3) Storage Qos
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> The admin needs to specify storage tags that only map
> to
> >>>>>>>> storage
> >>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>> supports Storage QoS.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> The admin needs to be aware for Hypervisor QoS that
> >> unless
> >>>>>> all
> >>>>>>>>>>>>>>>>>> hypervisors
> >>>>>>>>>>>>>>>>>>> in use support the new fields, they may not be
> enforced.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Post 4.3:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Come up with a way to more generally enter these
> >> parameters
> >>>>>>>>>>>>>> (probably
> >>>>>>>>>>>>>>>>>> just
> >>>>>>>>>>>>>>>>>>> key/value pairs sent to the drivers).
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Have the drivers expose their feature set so the
> >> allocators
> >>>>>> can
> >>>>>>>>>>>>>>>> consider
> >>>>>>>>>>>>>>>>>>> them more fully and throw an exception if there is not
> a
> >>>>>>>>>> sufficient
> >>>>>>>>>>>>>>>>>> match.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Thu, Jun 13, 2013 at 3:31 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> My thinking is, for 4.2, while not ideal, we will need
> >> to
> >>>>>> put
> >>>>>>>>>> some
> >>>>>>>>>>>>>>>>>> burden
> >>>>>>>>>>>>>>>>>>>> on the admin to configure a Disk Offering in a way
> that
> >>>>>> makes
> >>>>>>>>>>>>>> sense.
> >>>>>>>>>>>>>>>> For
> >>>>>>>>>>>>>>>>>>>> example, if he wants to use storage QoS with supported
> >> Min
> >>>>>> and
> >>>>>>>>>> Max
> >>>>>>>>>>>>>>>>>> values,
> >>>>>>>>>>>>>>>>>>>> he'll have to put in a storage tag that references the
> >>>>>>>> SolidFire
> >>>>>>>>>>>>>>>> primary
> >>>>>>>>>>>>>>>>>>>> storage (plug-in). If he puts in a storage tag that
> >>>> doesn't,
> >>>>>>>>>> then
> >>>>>>>>>>>>>> he's
> >>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>> going to get the Min and Max feature. We could add
> help
> >>>> text
> >>>>>>>> to
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> pop-up
> >>>>>>>>>>>>>>>>>>>> dialog that's displayed when you click in the Min and
> >> Max
> >>>>>> text
> >>>>>>>>>>>>>> fields.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Same idea for Wei's feature.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Not idea, true...perhaps we can brainstorm on a more
> >>>>>>>>>> comprehensive
> >>>>>>>>>>>>>>>>>>>> approach for post 4.2.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Maybe in the future we could have the drivers
> advertise
> >>>>>> their
> >>>>>>>>>>>>>>>>>> capabilities
> >>>>>>>>>>>>>>>>>>>> and if the allocator feels a request is not being
> >>>> satisfied
> >>>>>>>> (say
> >>>>>>>>>>>>>> Min
> >>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>> entered, but it not's supported by any storage
> plug-in)
> >> it
> >>>>>> can
> >>>>>>>>>>>>>> throw
> >>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>> exception.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Thu, Jun 13, 2013 at 3:19 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Comments below in red.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Thanks
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Thu, Jun 13, 2013 at 2:54 PM, John Burwell <
> >>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Please see my comment in-line below.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Jun 13, 2013, at 1:22 AM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Hi John,
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> I've put comments below in red.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:51 PM, John Burwell <
> >>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> First and foremost, we must ensure that these two
> >>>>>> features
> >>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>> mutually
> >>>>>>>>>>>>>>>>>>>>>>>> exclusive in 4.2. We don't want to find a
> >>>> configuration
> >>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>> contains both
> >>>>>>>>>>>>>>>>>>>>>>>> hypervisor and storage IOPS guarantees that leads
> to
> >>>>>>>>>>>>>>>>>> non-deterministic
> >>>>>>>>>>>>>>>>>>>>>>>> operations.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Agreed
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Restricting QoS expression to be either hypervisor
> >> or
> >>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>> oriented
> >>>>>>>>>>>>>>>>>>>>>>>> solves the problem in short term. As I understand
> >>>>>> storage
> >>>>>>>>>>>>>> tags,
> >>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>> have no
> >>>>>>>>>>>>>>>>>>>>>>>> means of expressing this type of mutual
> exclusion. I
> >>>>>>>> wasn't
> >>>>>>>>>>>>>>>>>>>>>> necessarily
> >>>>>>>>>>>>>>>>>>>>>>>> intending that we implement this allocation model
> in
> >>>>>> 4.3,
> >>>>>>>>>> but
> >>>>>>>>>>>>>>>>>> instead,
> >>>>>>>>>>>>>>>>>>>>>>>> determine if this type model would be one we would
> >>>> want
> >>>>>> to
> >>>>>>>>>>>>>> support
> >>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> future. If so, I would encourage us to ensure that
> >> the
> >>>>>>>> data
> >>>>>>>>>>>>>> model
> >>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>> current implementation would not preclude
> evolution
> >> in
> >>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>> direction. My
> >>>>>>>>>>>>>>>>>>>>>>>> view is that this type of allocation model is what
> >>>>>> user's
> >>>>>>>>>>>>>> expect
> >>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>> "cloud"
> >>>>>>>>>>>>>>>>>>>>>>>> systems -- selecting the best available resource
> set
> >>>> to
> >>>>>>>>>>>>>> fulfill a
> >>>>>>>>>>>>>>>>>>>>>> set of
> >>>>>>>>>>>>>>>>>>>>>>>> system requirements.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> I believe we have meet your requirement here in
> that
> >>>> what
> >>>>>>>>>> we've
> >>>>>>>>>>>>>>>>>>>>>> implemented
> >>>>>>>>>>>>>>>>>>>>>>> should not make refinement difficult in the future.
> >> If
> >>>> we
> >>>>>>>>>> don't
> >>>>>>>>>>>>>>>>>> modify
> >>>>>>>>>>>>>>>>>>>>>>> allocators for 4.2, but we do for 4.3, we've made
> >>>>>>>> relatively
> >>>>>>>>>>>>>> simple
> >>>>>>>>>>>>>>>>>>>>>> changes
> >>>>>>>>>>>>>>>>>>>>>>> to enhance the current functioning of the system.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Looking through both patches, I have to say that the
> >>>>>>>>>> aggregated
> >>>>>>>>>>>>>>>> result
> >>>>>>>>>>>>>>>>>>>>>> seems a bit confusing. There are six new attributes
> >> for
> >>>>>>>>>>>>>> throttled
> >>>>>>>>>>>>>>>>>> I/O and
> >>>>>>>>>>>>>>>>>>>>>> two for provisioned IOPS with no obvious grouping.
> My
> >>>>>>>> concern
> >>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>> technical, but rather, about maintainability. At
> >>>> minimum,
> >>>>>>>>>>>>>> Javadoc
> >>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>> be added explaining the two sets of attributes and
> >> their
> >>>>>>>>>> mutual
> >>>>>>>>>>>>>>>>>> exclusion.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I agree: We need JavaDoc to explain them and their
> >> mutual
> >>>>>>>>>>>>>> exclusion.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> The other part that is interesting is that throttled
> >> I/O
> >>>>>>>>>>>>>> provides
> >>>>>>>>>>>>>>>> both
> >>>>>>>>>>>>>>>>>>>>>> an IOPS and byte measured QoS as a rate where
> >>>> provisioned
> >>>>>>>> IOPS
> >>>>>>>>>>>>>> uses
> >>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>> range. In order to select the best available
> resource
> >> to
> >>>>>>>>>>>>>> fulfill a
> >>>>>>>>>>>>>>>>>> QoS, we
> >>>>>>>>>>>>>>>>>>>>>> would need to have the QoS expression normalized in
> >>>> terms
> >>>>>> of
> >>>>>>>>>>>>>> units
> >>>>>>>>>>>>>>>>>> (IOPS or
> >>>>>>>>>>>>>>>>>>>>>> bytes) and their expression (rate vs. range). If we
> >> want
> >>>>>> to
> >>>>>>>>>>>>>>>> achieve a
> >>>>>>>>>>>>>>>>>>>>>> model like I described, I think we would need to
> >> resolve
> >>>>>>>> this
> >>>>>>>>>>>>>> issue
> >>>>>>>>>>>>>>>>>> in 4.2
> >>>>>>>>>>>>>>>>>>>>>> to ensure a viable migration path.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I think we're not likely to be able to normalize the
> >>>> input
> >>>>>>>> for
> >>>>>>>>>>>>>> 4.2.
> >>>>>>>>>>>>>>>>>> Plus
> >>>>>>>>>>>>>>>>>>>>> people probably want to input the data in terms
> they're
> >>>>>>>>>> familiar
> >>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>> the products in question.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Ideally we would fix the way we do storage tagging
> and
> >>>> let
> >>>>>>>> the
> >>>>>>>>>>>>>> user
> >>>>>>>>>>>>>>>>>> send
> >>>>>>>>>>>>>>>>>>>>> key/value pairs to each vendor that could be selected
> >> due
> >>>>>> to
> >>>>>>>> a
> >>>>>>>>>>>>>> given
> >>>>>>>>>>>>>>>>>>>>> storage tag. I'm still not sure that would solve it
> >>>> because
> >>>>>>>>>> what
> >>>>>>>>>>>>>>>>>> happens if
> >>>>>>>>>>>>>>>>>>>>> you change the storage tag of a given Primary Storage
> >>>> after
> >>>>>>>>>>>>>> having
> >>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>> a Disk Offering?
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Basically storage tagging is kind of a mess and we
> >> should
> >>>>>>>>>>>>>> re-think
> >>>>>>>>>>>>>>>> it.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Also, we need to have a way for the drivers to expose
> >>>> their
> >>>>>>>>>>>>>> supported
> >>>>>>>>>>>>>>>>>>>>> feature sets so the allocators can make good choices.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> As I think through the implications of these
> >>>>>> requirements
> >>>>>>>>>> and
> >>>>>>>>>>>>>>>>>> reflect
> >>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>> the reviews, I don't understand why they haven't
> >>>> already
> >>>>>>>>>>>>>> impacted
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> allocators and planners. As it stands, the current
> >>>>>>>>>>>>>> provisioned
> >>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>> has no
> >>>>>>>>>>>>>>>>>>>>>>>> checks to ensure that the volumes are allocated to
> >>>>>> devices
> >>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>> capacity to fulfill the requested QoS. Therefore,
> >> as I
> >>>>>>>>>>>>>> understand
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> current patch, we can overcommit storage resources
> >> --
> >>>>>>>>>>>>>> potentially
> >>>>>>>>>>>>>>>>>>>>>> causing
> >>>>>>>>>>>>>>>>>>>>>>>> none of the QoS obligations from being fulfilled.
> It
> >>>>>> seems
> >>>>>>>>>>>>>> to me
> >>>>>>>>>>>>>>>>>>>>>> that a
> >>>>>>>>>>>>>>>>>>>>>>>> DataStore supporting provisioned IOPS should
> express
> >>>> the
> >>>>>>>>>>>>>> maximum
> >>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>>>>>>>>> it can support and some type of overcommitment
> >> factor.
> >>>>>>>> This
> >>>>>>>>>>>>>>>>>>>>>> information
> >>>>>>>>>>>>>>>>>>>>>>>> should be used by the storage allocators to
> >> determine
> >>>>>> the
> >>>>>>>>>>>>>> device
> >>>>>>>>>>>>>>>>>> best
> >>>>>>>>>>>>>>>>>>>>>> able
> >>>>>>>>>>>>>>>>>>>>>>>> to support the resources needs of a volume. It
> seems
> >>>>>> that
> >>>>>>>> a
> >>>>>>>>>>>>>>>> similar
> >>>>>>>>>>>>>>>>>>>>>> set of
> >>>>>>>>>>>>>>>>>>>>>>>> considerations would need to be added to the
> >>>> Hypervisor
> >>>>>>>>>> layer
> >>>>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>>>>>>>>> allocating a VM to a host to prevent
> >> oversubscription.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Yeah, for this first release, we just followed the
> >> path
> >>>>>>>> that
> >>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>> previously
> >>>>>>>>>>>>>>>>>>>>>>> established for other properties you see on dialogs
> >> in
> >>>>>> CS:
> >>>>>>>>>> Just
> >>>>>>>>>>>>>>>>>> because
> >>>>>>>>>>>>>>>>>>>>>>> they're there doesn't mean the vendor your VM is
> >>>> deployed
> >>>>>>>> to
> >>>>>>>>>>>>>>>> supports
> >>>>>>>>>>>>>>>>>>>>>> them.
> >>>>>>>>>>>>>>>>>>>>>>> It is then up to the admin to make sure he inputs,
> >>>> say, a
> >>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>> tag
> >>>>>>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>>>>> confines the deployment only to storage that
> supports
> >>>> the
> >>>>>>>>>>>>>> selected
> >>>>>>>>>>>>>>>>>>>>>>> features. This is not ideal, but it's kind of the
> way
> >>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>>>>> works
> >>>>>>>>>>>>>>>>>>>>>>> today.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> I understand the tag functionality, and the need for
> >> the
> >>>>>>>>>>>>>>>> administrator
> >>>>>>>>>>>>>>>>>>>>>> to very carefully construct offerings. My concern is
> >>>> that
> >>>>>> we
> >>>>>>>>>>>>>> over
> >>>>>>>>>>>>>>>>>>>>>> guarantee a resource's available IOPS. For the
> >> purposes
> >>>> of
> >>>>>>>>>>>>>>>>>> illustration,
> >>>>>>>>>>>>>>>>>>>>>> let's say we have a SolidFire, and the max IOPS for
> >> that
> >>>>>>>>>> device
> >>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>> 100000.
> >>>>>>>>>>>>>>>>>>>>>> We also know that we can safely oversubscribe by
> 50%.
> >>>>>>>>>>>>>> Therefore,
> >>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>> need to ensure that we don't allocate more than
> >> 150,000
> >>>>>>>>>>>>>> guaranteed
> >>>>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>>>>>> from that device. Intuitively, it seems like the
> >>>> DataStore
> >>>>>>>>>>>>>>>>>> configuration
> >>>>>>>>>>>>>>>>>>>>>> should have a max assignable IOPS and overcommitment
> >>>>>> factor.
> >>>>>>>>>>>>>> As we
> >>>>>>>>>>>>>>>>>>>>>> allocate volumes and attach VMs, we need to ensure
> >> that
> >>>>>>>>>>>>>> guarantee
> >>>>>>>>>>>>>>>>>> more IOPS
> >>>>>>>>>>>>>>>>>>>>>> exceed the configured maximum for a DataStore. Does
> >> that
> >>>>>>>> make
> >>>>>>>>>>>>>>>> sense?
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I think that's a good idea for a future enhancement.
> >> I'm
> >>>>>> not
> >>>>>>>>>> even
> >>>>>>>>>>>>>>>> sure
> >>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>> can query the SAN to find out how many IOPS safely
> >>>> remain.
> >>>>>>>> I'd
> >>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>> get
> >>>>>>>>>>>>>>>>>>>>> all of the min values for all of the volumes on the
> SAN
> >>>> and
> >>>>>>>>>> total
> >>>>>>>>>>>>>>>> them
> >>>>>>>>>>>>>>>>>> up,
> >>>>>>>>>>>>>>>>>>>>> I suppose, and subtract it from the total (user
> facing)
> >>>>>>>>>> supported
> >>>>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>> the system.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Another question occurs to me -- should we allow
> >>>> non-QoS
> >>>>>>>>>>>>>> resources
> >>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>> assigned to hosts/storage devices that ensure QoS?
> >> For
> >>>>>>>>>>>>>>>> provisioned
> >>>>>>>>>>>>>>>>>>>>>> IOPS, I
> >>>>>>>>>>>>>>>>>>>>>>>> think a side effect of the current implementation
> is
> >>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>> volumes
> >>>>>>>>>>>>>>>>>>>>>>>> always have a QoS. However, for hypervisor
> throttled
> >>>>>> I/O,
> >>>>>>>> it
> >>>>>>>>>>>>>>>> seems
> >>>>>>>>>>>>>>>>>>>>>>>> entirely possible for non-QoS VMs to allocated
> >>>>>>>> side-by-side
> >>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>> QoS
> >>>>>>>>>>>>>>>>>>>>>> VMs.
> >>>>>>>>>>>>>>>>>>>>>>>> In this scenario, a greedy, unbounded VM could
> >>>>>> potentially
> >>>>>>>>>>>>>> starve
> >>>>>>>>>>>>>>>>>> out
> >>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> other VMs on the host -- preventing the QoSes
> >> defined
> >>>>>> the
> >>>>>>>>>>>>>>>> collocated
> >>>>>>>>>>>>>>>>>>>>>> VMs
> >>>>>>>>>>>>>>>>>>>>>>>> from being fulfilled.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> You can make SolidFire volumes (inside and outside
> of
> >>>> CS)
> >>>>>>>> and
> >>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>> specify
> >>>>>>>>>>>>>>>>>>>>>>> IOPS. You'll still get guaranteed IOPS, but it will
> >> be
> >>>> at
> >>>>>>>> the
> >>>>>>>>>>>>>>>>>> defaults
> >>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>> choose. Unless you over-provision IOPS on a
> SolidFire
> >>>>>> SAN,
> >>>>>>>>>> you
> >>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>> your Mins met.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> It sounds like you're perhaps looking for a storage
> >>>> tags
> >>>>>>>>>>>>>> exclusions
> >>>>>>>>>>>>>>>>>>>>>> list,
> >>>>>>>>>>>>>>>>>>>>>>> which might be nice to have at some point (i.e.
> don't
> >>>>>>>> deploy
> >>>>>>>>>> my
> >>>>>>>>>>>>>>>>>> volume
> >>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>> storage with these following tags).
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> I don't like the idea of a storage tags exclusion
> list
> >>>> as
> >>>>>> it
> >>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>>>>>>> complicate component assembly. It would require a
> >>>> storage
> >>>>>>>>>>>>>> plugin to
> >>>>>>>>>>>>>>>>>>>>>> anticipate all of the possible component assemblies
> >> and
> >>>>>>>>>>>>>> determine
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> invalid relationships. I prefer that drivers express
> >>>> their
> >>>>>>>>>>>>>>>>>> capabilities
> >>>>>>>>>>>>>>>>>>>>>> which can be matched to a set of requested
> >> requirements.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I'm not sure why a storage plug-in would care about
> >>>>>> inclusion
> >>>>>>>>>> or
> >>>>>>>>>>>>>>>>>>>>> exclusion lists. It just needs to advertise its
> >>>>>> functionality
> >>>>>>>>>> in
> >>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>> way
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> allocator understands.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> I agree with your assessment of Hypervisor QoS.
> Since
> >>>> it
> >>>>>>>> only
> >>>>>>>>>>>>>>>> limits
> >>>>>>>>>>>>>>>>>>>>>> IOPS,
> >>>>>>>>>>>>>>>>>>>>>>> it does not solve the Noisy Neighbor problem. Only
> a
> >>>>>> system
> >>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>> guaranteed
> >>>>>>>>>>>>>>>>>>>>>>> minimum IOPS does this.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> As I said, for SolidFire, it sounds like this
> problem
> >>>> does
> >>>>>>>> not
> >>>>>>>>>>>>>>>> exist.
> >>>>>>>>>>>>>>>>>>>>>> However, I am concerned with the more general case
> as
> >> we
> >>>>>>>>>>>>>> supported
> >>>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>> devices with provisioned IOPS.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Post 4.2 we need to investigate a way to pass
> >>>>>> vendor-specific
> >>>>>>>>>>>>>> values
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>> drivers. Min and Max and pretty industry standard for
> >>>>>>>>>> provisioned
> >>>>>>>>>>>>>>>>>> IOPS, but
> >>>>>>>>>>>>>>>>>>>>> what if you break them out by read and write or do
> >>>>>> something
> >>>>>>>>>>>>>> else? We
> >>>>>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>>>>>> a more general mechanism.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> In my opinion, we need to ensure that hypervisor
> >>>>>> throttled
> >>>>>>>>>>>>>> I/O
> >>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>> storage provisioned IOPS are mutually exclusive
> per
> >>>>>>>> volume.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Agreed
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> We also need to understand the implications of
> these
> >>>> QoS
> >>>>>>>>>>>>>>>> guarantees
> >>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>> operation of the system to ensure that the
> >> underlying
> >>>>>>>>>> hardware
> >>>>>>>>>>>>>>>>>>>>>> resources
> >>>>>>>>>>>>>>>>>>>>>>>> can fulfill them. Given the time frame, we will
> >> likely
> >>>>>> be
> >>>>>>>>>>>>>> forced
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>>>>>> compromises to achieve these goals, and refine the
> >>>>>>>>>>>>>> implementation
> >>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>> 4.3.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> I agree, John. I also think you've come up with
> some
> >>>>>> great
> >>>>>>>>>>>>>> ideas
> >>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>> 4.3. :)
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 11:35 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Yeah, Alex, I think that's the way we were
> planning
> >>>>>> (with
> >>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>> tags).
> >>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>> believe John was just throwing out an idea that -
> >> in
> >>>>>>>>>>>>>> addition to
> >>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>> tags - we could look into these allocators
> (storage
> >>>> QoS
> >>>>>>>>>> being
> >>>>>>>>>>>>>>>>>>>>>> preferred,
> >>>>>>>>>>>>>>>>>>>>>>>>> then hypervisor QoS if storage QoS is not
> >> available,
> >>>>>> but
> >>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>> QoS
> >>>>>>>>>>>>>>>>>>>>>>>> is).
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> I think John's concern is that you can enter in
> >>>> values
> >>>>>>>> for
> >>>>>>>>>>>>>> Wei's
> >>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>>>>>>>>>>> feature that are not honored by other vendors (at
> >>>> least
> >>>>>>>>>>>>>> yet), so
> >>>>>>>>>>>>>>>> he
> >>>>>>>>>>>>>>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>>>>>>>> hoping - in addition to storage tags - for the
> >>>>>> allocators
> >>>>>>>>>> to
> >>>>>>>>>>>>>>>> prefer
> >>>>>>>>>>>>>>>>>>>>>> these
> >>>>>>>>>>>>>>>>>>>>>>>>> vendors when these fields are filled in. As it
> >> stands
> >>>>>>>> today
> >>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>> CloudStack,
> >>>>>>>>>>>>>>>>>>>>>>>>> we already have this kind of an issue with other
> >>>>>> features
> >>>>>>>>>>>>>> (fields
> >>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>> dialogs for features that not all vendors
> support).
> >>>>>>>> Perhaps
> >>>>>>>>>>>>>> post
> >>>>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>> could look into generic name/value pairs (this is
> >> how
> >>>>>>>>>>>>>> OpenStack
> >>>>>>>>>>>>>>>>>>>>>> addresses
> >>>>>>>>>>>>>>>>>>>>>>>>> the issue).
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Honestly, I think we're too late in the game (two
> >>>> weeks
> >>>>>>>>>> until
> >>>>>>>>>>>>>>>> code
> >>>>>>>>>>>>>>>>>>>>>>>> freeze)
> >>>>>>>>>>>>>>>>>>>>>>>>> to go too deeply down that path in 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> It's probably better if we - at least for 4.2 -
> >> keep
> >>>>>>>> Wei's
> >>>>>>>>>>>>>> fields
> >>>>>>>>>>>>>>>>>>>>>> and my
> >>>>>>>>>>>>>>>>>>>>>>>>> fields as is, make sure only one or the other
> >> feature
> >>>>>> has
> >>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>> entered
> >>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>> it (or neither), and call it good for 4.2.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Then let's step back and look into a more
> >>>>>> general-purpose
> >>>>>>>>>>>>>> design
> >>>>>>>>>>>>>>>>>>>>>> that can
> >>>>>>>>>>>>>>>>>>>>>>>>> be applied throughout CloudStack where we have
> >> these
> >>>>>>>>>>>>>> situations.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 5:21 PM, John Burwell <
> >>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> I just published my review @
> >>>>>>>>>>>>>>>> https://reviews.apache.org/r/11479/.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> I apologize for the delay,
> >>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> No problem, John.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> I still want to re-review it by myself before
> >>>> coming
> >>>>>> up
> >>>>>>>>>>>>>> with a
> >>>>>>>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>>>>>>>>> patch
> >>>>>>>>>>>>>>>>>>>>>>>>>>> file.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Also, maybe I should first wait for Wei's
> changes
> >>>> to
> >>>>>> be
> >>>>>>>>>>>>>> checked
> >>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>> merge those into mine before generating a new
> >> patch
> >>>>>>>> file?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell
> <
> >>>>>>>>>>>>>>>>>> jburwell@basho.com
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I just realized that I forgot to publish my
> >>>> review.
> >>>>>> I
> >>>>>>>> am
> >>>>>>>>>>>>>>>>>> offline
> >>>>>>>>>>>>>>>>>>>>>> ATM,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> but I will publish it in the next couple of
> >> hours.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Do you plan to update your the patch in Review
> >>>>>> Board?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry for the oversight,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> <mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is
> >>>>>> reading
> >>>>>>>>>>>>>> this :)
> >>>>>>>>>>>>>>>>>> ),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Just an FYI that I believe I have implemented
> >> all
> >>>>>> the
> >>>>>>>>>>>>>> areas
> >>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>> wanted
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> addressed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I plan to review the code again tomorrow
> >> morning
> >>>> or
> >>>>>>>>>>>>>>>> afternoon,
> >>>>>>>>>>>>>>>>>>>>>> then
> >>>>>>>>>>>>>>>>>>>>>>>>>> send
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> another patch.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for all the work on this everyone!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike
> >> Tutkowski
> >>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sure, that sounds good.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU <
> >>>>>>>>>>>>>>>>>>>>>> ustcweizhou@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It looks the two feature do not have many
> >>>>>> conflicts
> >>>>>>>>>> in
> >>>>>>>>>>>>>> Java
> >>>>>>>>>>>>>>>>>>>>>> code,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> except
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the cloudstack UI.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you do not mind, I will merge
> >>>>>> disk_io_throttling
> >>>>>>>>>>>>>> branch
> >>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>> master
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> week, so that you can develop based on it.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Wei
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <
> >>>>>>>>>> mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hey John,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The SolidFire patch does not depend on the
> >>>>>>>>>>>>>> object_store
> >>>>>>>>>>>>>>>>>>>>>> branch,
> >>>>>>>>>>>>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>>>>>>>>>>>> -
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Edison mentioned - it might be easier if
> we
> >>>>>> merge
> >>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>>>>>> branch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the object_store branch before
> object_store
> >>>> goes
> >>>>>>>>>> into
> >>>>>>>>>>>>>>>>>> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling
> fits
> >>>>>> into
> >>>>>>>>>> this
> >>>>>>>>>>>>>>>> merge
> >>>>>>>>>>>>>>>>>>>>>>>>>> strategy.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps Wei can chime in on that.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John
> >>>> Burwell <
> >>>>>>>>>>>>>>>>>>>>>>>> jburwell@basho.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mike,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We have a delicate merge dance to
> perform.
> >>>> The
> >>>>>>>>>>>>>>>>>>>>>>>> disk_io_throttling,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> solidfire, and object_store appear to
> have
> >> a
> >>>>>>>> number
> >>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>> overlapping
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> elements. I understand the dependencies
> >>>> between
> >>>>>>>> the
> >>>>>>>>>>>>>>>>>> patches
> >>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> follows:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> object_store <- solidfire ->
> >>>> disk_io_throttling
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Am I correct that the device management
> >>>> aspects
> >>>>>>>> of
> >>>>>>>>>>>>>>>>>> SolidFire
> >>>>>>>>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> additive
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to the object_store branch or there are
> >>>>>> circular
> >>>>>>>>>>>>>>>> dependency
> >>>>>>>>>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> branches? Once we understand the
> dependency
> >>>>>>>> graph,
> >>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>> determine
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> best approach to land the changes in
> >> master.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -John
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike
> >> Tutkowski
> >>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging
> >> my
> >>>>>> code
> >>>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>> his
> >>>>>>>>>>>>>>>>>>>>>> branch
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> before
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> going into master, I am good with that.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We can remove the
> StoragePoolType.Dynamic
> >>>> code
> >>>>>>>>>>>>>> after his
> >>>>>>>>>>>>>>>>>>>>>> merge
> >>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike
> >>>>>> Tutkowski
> >>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're
> >> going
> >>>>>>>> here:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1) There should be NO references to
> >>>>>> hypervisor
> >>>>>>>>>>>>>> code in
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> storage
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> plug-ins code (this includes the
> default
> >>>>>>>> storage
> >>>>>>>>>>>>>>>> plug-in,
> >>>>>>>>>>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> currently
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sends several commands to the
> hypervisor
> >> in
> >>>>>> use
> >>>>>>>>>>>>>>>> (although
> >>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>> does
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> know
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i),
> >> etc.)
> >>>> is
> >>>>>>>>>>>>>> actually
> >>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>> use))
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be
> >>>>>> placed
> >>>>>>>> in
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>> url
> >>>>>>>>>>>>>>>>>>>>>> field
> >>>>>>>>>>>>>>>>>>>>>>>>>> (if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> present, we default to false). This
> info
> >> is
> >>>>>>>>>> stored
> >>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> storage_pool_details table.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to
> >> the
> >>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> question, we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pass the managed property along (this
> >> takes
> >>>>>> the
> >>>>>>>>>>>>>> place
> >>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the
> >>>>>>>> hypervisor
> >>>>>>>>>>>>>>>> checks
> >>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> managed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> property. If true for an attach, the
> >>>>>> necessary
> >>>>>>>>>>>>>>>> hypervisor
> >>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> structure is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created and the rest of the attach
> >> command
> >>>>>>>>>>>>>> executes to
> >>>>>>>>>>>>>>>>>>>>>> attach
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> volume.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is
> >>>>>> invoked
> >>>>>>>>>> to
> >>>>>>>>>>>>>>>>>> detach a
> >>>>>>>>>>>>>>>>>>>>>>>>>> volume,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> same check is made. If managed, the
> >>>>>> hypervisor
> >>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>> structure
> >>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> removed.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support
> >>>> Burst
> >>>>>>>>>> IOPS
> >>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>> 4.2
> >>>>>>>>>>>>>>>>>>>>>>>> unless
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stored in the volumes and
> disk_offerings
> >>>>>> table.
> >>>>>>>>>> If
> >>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>> some
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> idea,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that'd be cool.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike
> >>>>>>>> Tutkowski <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented
> >> while
> >>>>>>>>>>>>>> avoiding
> >>>>>>>>>>>>>>>>>>>>>>>>>> implementation
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attributes. I always wondered about
> the
> >>>>>>>> details
> >>>>>>>>>>>>>>>> field.
> >>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> beef up the description in the
> >>>> documentation
> >>>>>>>>>>>>>> regarding
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> expected
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> format
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that
> the
> >>>>>>>> details
> >>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> returned on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool,
> or
> >>>>>>>>>>>>>>>> listStoragePool
> >>>>>>>>>>>>>>>>>>>>>>>>>> response.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Why
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it
> >> would
> >>>>>> be
> >>>>>>>>>>>>>> useful
> >>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>> clients
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> able to inspect the contents of the
> >>>> details
> >>>>>>>>>>>>>> field."
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Not sure how this would work storing
> >> Burst
> >>>>>>>> IOPS
> >>>>>>>>>>>>>> here.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a
> Disk
> >>>>>>>>>>>>>>>>>> Offering-by-Disk
> >>>>>>>>>>>>>>>>>>>>>>>>>> Offering
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created,
> >> you
> >>>>>>>> have
> >>>>>>>>>>>>>> to be
> >>>>>>>>>>>>>>>>>>>>>> able to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> associate
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a
> >>>>>>>>>>>>>> disk_offering_details
> >>>>>>>>>>>>>>>>>> table.
> >>>>>>>>>>>>>>>>>>>>>>>> Maybe
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> go there?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept
> >> the
> >>>>>>>> Burst
> >>>>>>>>>>>>>> IOPS
> >>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> GUI
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields
> >> are
> >>>>>> in
> >>>>>>>>>> the
> >>>>>>>>>>>>>> DB.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >> cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *™*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >>>> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *™*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> >> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *™*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
> Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> *™*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> *™*
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> *™*
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> *™*
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>>>>>> cloud<
> >>>> http://solidfire.com/solution/overview/?video=play
> >>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> *™*
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>>> *™*
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>>>>>>> *™*
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>>>> cloud<
> http://solidfire.com/solution/overview/?video=play
> >>>
> >>>>>>>>>>>>>>>>>>> *™*
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play
> >
> >>>>>>>>>>>>>>>>> *™*
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>>>> *™*
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>>> *™*
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> --
> >>>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>>> Advancing the way the world uses the cloud<
> >>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>>> *™*
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> *Mike Tutkowski*
> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>>>> o: 303.746.7302
> >>>>>>>>>>> Advancing the way the world uses the
> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>>>> *™*
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> *Mike Tutkowski*
> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>>>> o: 303.746.7302
> >>>>>>>>> Advancing the way the world uses the
> >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>>>> *™*
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> *Mike Tutkowski*
> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>>>> e: mike.tutkowski@solidfire.com
> >>>>>>> o: 303.746.7302
> >>>>>>> Advancing the way the world uses the
> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>>>> *™*
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> *Mike Tutkowski*
> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> >>>>> e: mike.tutkowski@solidfire.com
> >>>>> o: 303.746.7302
> >>>>> Advancing the way the world uses the
> >>>>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>>>> *™*
> >>>>
> >>>>
> >>>
> >>>
> >>> --
> >>> *Mike Tutkowski*
> >>> *Senior CloudStack Developer, SolidFire Inc.*
> >>> e: mike.tutkowski@solidfire.com
> >>> o: 303.746.7302
> >>> Advancing the way the world uses the
> >>> cloud<http://solidfire.com/solution/overview/?video=play>
> >>> *™*
> >>
> >>
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *™*
>
>


-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message