cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Burwell <jburw...@basho.com>
Subject Re: [MERGE] disk_io_throttling to MASTER (Second Round)
Date Mon, 10 Jun 2013 22:52:13 GMT
Mike,

From a CloudStack perspective, it will keep implementation specific concepts from the base
data model, and provide a great test case to develop a mechanism to capture this information
in 4.3.  Ideally, I want CloudStack to exploit these implementation specific features.  I
just want to provide a facility to manage that data that does not leak across implementations.

Thanks,
-John

On Jun 10, 2013, at 6:07 PM, Mike Tutkowski <mike.tutkowski@solidfire.com> wrote:

> I am pretty sure it is not a blocker for me to drop Burst IOPS. Let me run
> that past Product Management, but I don't think it's a problem.
> 
> 
> On Mon, Jun 10, 2013 at 4:01 PM, John Burwell <jburwell@basho.com> wrote:
> 
>> Mike,
>> 
>> Yes.  I realize my other reply did not explicitly state leaving the Min
>> and Max IOPS fields in the data model as these seem to generic terms across
>> storage implementations.
>> 
>> Thanks,
>> -John
>> 
>> On Jun 10, 2013, at 5:57 PM, Mike Tutkowski <mike.tutkowski@solidfire.com>
>> wrote:
>> 
>>> More generally speaking, you're looking to remove Burst IOPS from
>>> CloudStack for 4.2, but we would keep Min and Max (and they would be
>>> displayed in the Disk Offering dialog as I've proposed)?
>>> 
>>> 
>>> On Mon, Jun 10, 2013 at 3:52 PM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com> wrote:
>>> 
>>>> Just to make sure I understand your request, are you looking to display
>>>> Min and Max (as long as Wei's feature is not in use), but not display
>> Burst
>>>> IOPS?
>>>> 
>>>> 
>>>> On Mon, Jun 10, 2013 at 3:50 PM, John Burwell <jburwell@basho.com>
>> wrote:
>>>> 
>>>>> Mike,
>>>>> 
>>>>> My concern becomes that we start ballooning the data model and user
>>>>> interface with a fields that are documented as, "If using SolidFire,
>> then
>>>>> Burst IOPS is honored and foo and bar are not.  For solution X, Burst
>> IOPS
>>>>> is ignored, but foo and bar apply."  It may have to hold until 4.3,
>> but it
>>>>> seems like we need an extended data concept for storage drivers that
>> allow
>>>>> them to define an additional set of properties, and persist them into
>> the
>>>>> database as a JSON document.  Such an enhancement would also require
>> some
>>>>> UI fanciness to consume the metadata provided by the driver and adjust
>> the
>>>>> UI.  Would it be possible to defer Burst IOPS until 4.3 when we could
>>>>> address extended driver data in a holistic manner?
>>>>> 
>>>>> Thanks,
>>>>> -John
>>>>> 
>>>>> On Jun 10, 2013, at 4:44 PM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com>
>>>>> wrote:
>>>>> 
>>>>>> My thinking is that Min and Max are industry standard and Burst is
a
>> new
>>>>>> concept that could catch on.
>>>>>> 
>>>>>> 
>>>>>> On Mon, Jun 10, 2013 at 2:29 PM, John Burwell <jburwell@basho.com>
>>>>> wrote:
>>>>>> 
>>>>>>> Wei,
>>>>>>> 
>>>>>>> In this case, we can have the hypervisor or storage providing
the
>>>>> quality
>>>>>>> of service guarantee.  Naively, it seems reasonable to separate
>>>>> hypervisor
>>>>>>> and storage QoS parameters and enforce mutually exclusion.  Not
only
>>>>> does
>>>>>>> this approach simplify a whole range of operations, it also avoids
>>>>>>> reconciling the data models.  The only question I have about
the
>>>>> storage
>>>>>>> provision IOPS is whether or not "Burst IOPS" is an industry
standard
>>>>> or
>>>>>>> vendor specific concept.
>>>>>>> 
>>>>>>> Thanks,
>>>>>>> -John
>>>>>>> 
>>>>>>> On Jun 10, 2013, at 3:59 PM, Wei ZHOU <ustcweizhou@gmail.com>
wrote:
>>>>>>> 
>>>>>>>> Mike,
>>>>>>>> I do not think users can select only one of them, as they
are
>>>>> implemented
>>>>>>>> on different sides.
>>>>>>>> Have you investigated the parameters other storage devices
support,
>>>>>>> besides
>>>>>>>> min/max/burst IOPS? You'd better add all possible fields
in your
>>>>>>>> implementation.
>>>>>>>> 
>>>>>>>> What do you think about this?
>>>>>>>> Hypersivor IOPS is fixed, and there is a drop-down box which
>> includes
>>>>> all
>>>>>>>> supported storage vendors.
>>>>>>>> If users select "SolidFire", min/max/burst IOPS will appear.
>>>>>>>> If users select other vendors, relevant fields will appear.
>>>>>>>> Actually I still insist that it is better to add the storage-related
>>>>>>> fields
>>>>>>>> in another table.
>>>>>>>> 
>>>>>>>> -Wei
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 2013/6/10 Mike Tutkowski <mike.tutkowski@solidfire.com>
>>>>>>>> 
>>>>>>>>> Here is my thinking:
>>>>>>>>> 
>>>>>>>>> Two radio buttons (whatever we want to call them):
>>>>>>>>> 
>>>>>>>>> 1) Hypervisor IOPS
>>>>>>>>> 2) Storage IOPS
>>>>>>>>> 
>>>>>>>>> Leave them both un-checked by default.
>>>>>>>>> 
>>>>>>>>> If the user checks one or the other, the relevant fields
appear.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On Mon, Jun 10, 2013 at 1:38 PM, Mike Tutkowski <
>>>>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>>>> 
>>>>>>>>>> What do you think, Wei?
>>>>>>>>>> 
>>>>>>>>>> Should we come up with a way for only one feature
(yours or mine)
>>>>> to be
>>>>>>>>>> used at a time on the new Disk Offering dialog?
>>>>>>>>>> 
>>>>>>>>>> Since most storage-side provisioned IOPS don't break
it down into
>>>>>>>>> separate
>>>>>>>>>> read and write categories, I think that's the way
to go (only one
>>>>>>> feature
>>>>>>>>>> or the other).
>>>>>>>>>> 
>>>>>>>>>> Any suggestions from a usability standpoint how we
want to
>> implement
>>>>>>>>> this?
>>>>>>>>>> It could be as simple as a radio button to turn on
your feature
>> and
>>>>>>> mine
>>>>>>>>>> off or vice versa.
>>>>>>>>>> 
>>>>>>>>>> Thanks!
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> On Mon, Jun 10, 2013 at 1:33 PM, John Burwell <jburwell@basho.com
>>> 
>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> Mike,
>>>>>>>>>>> 
>>>>>>>>>>> I agree -- I can't image a situation where you
would want to use
>>>>> IOPS
>>>>>>>>>>> provisioned by both the hypervisor and storage.
 There are two
>>>>> points
>>>>>>> of
>>>>>>>>>>> concern -- the UI and the management server.
 We have to ensure
>>>>> that
>>>>>>> the
>>>>>>>>>>> user can't create a VM from a compute/disk offering
combination
>>>>> where
>>>>>>>>>>> hypervisor throttled I/O would contradict/conflict
with storage
>>>>>>>>> provisioned
>>>>>>>>>>> IOPS.  I think this functional conflict must
be resolved in the
>>>>>>>>> management
>>>>>>>>>>> server to ensure that API calls are properly
validated with a UX
>>>>> that
>>>>>>>>>>> avoids user confusion.  Have Wei and you worked
out an approach
>> to
>>>>>>>>>>> resolving this conflict?
>>>>>>>>>>> 
>>>>>>>>>>> Thanks,
>>>>>>>>>>> -John
>>>>>>>>>>> 
>>>>>>>>>>> On Jun 10, 2013, at 3:24 PM, Mike Tutkowski <
>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> Wei has sent me the screen shots.
>>>>>>>>>>>> 
>>>>>>>>>>>> I don't support Compute Offerings for 4.2,
so that's not an
>> issue
>>>>>>>>> here.
>>>>>>>>>>>> 
>>>>>>>>>>>> I do support Disk Offerings.
>>>>>>>>>>>> 
>>>>>>>>>>>> It looks like Wei has added four new fields
to the Disk
>> Offering.
>>>>>>>>>>>> 
>>>>>>>>>>>> I have added three (Min, Max, and Burst IOPS).
>>>>>>>>>>>> 
>>>>>>>>>>>> We just need to decide if we should toggle
between his and mine.
>>>>>>>>>>>> 
>>>>>>>>>>>> I doubt a user would want to use both features
at the same time.
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> On Mon, Jun 10, 2013 at 12:30 PM, John Burwell
<
>>>>> jburwell@basho.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> Mike,
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Have Wei and you figured out the system
level as well (e.g.
>>>>> allowing
>>>>>>>>>>>>> either storage provisioned IOPS or hypervisor
throttling, but
>> no
>>>>>>>>> both)?
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> -John
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Jun 10, 2013, at 2:12 PM, Mike Tutkowski
<
>>>>>>>>>>> mike.tutkowski@solidfire.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Perhaps Wei could send me some screen
shots of what he's
>>>>> changed in
>>>>>>>>>>> the
>>>>>>>>>>>>> GUI
>>>>>>>>>>>>>> for his feature?
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 11:56 AM,
John Burwell <
>>>>> jburwell@basho.com
>>>>>>>> 
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Wei,
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Have Mike Tutkowski and you reconciled
the potential conflict
>>>>>>>>>>> between a
>>>>>>>>>>>>>>> throttled I/O VM and a provisioned
IOPs volume?  If so, what
>>>>>>>>> solution
>>>>>>>>>>>>> did
>>>>>>>>>>>>>>> you select?
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> -John
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On Jun 10, 2013, at 1:54 PM,
Wei ZHOU <ustcweizhou@gmail.com
>>> 
>>>>>>>>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Guys,
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> I would like to merge disk_io_throttling
branch into master.
>>>>>>>>>>>>>>>> Please review the code on
>> https://reviews.apache.org/r/11782
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> If nobody object, I will
merge into master in 72 hours.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> -Wei
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 2013/5/30 Wei ZHOU <ustcweizhou@gmail.com>
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>> I would like to merge
disk_io_throttling branch into
>> master.
>>>>>>>>>>>>>>>>> If nobody object, I will
merge into master in 48 hours.
>>>>>>>>>>>>>>>>> The purpose is :
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Virtual machines are
running on the same storage device
>>>>> (local
>>>>>>>>>>> storage
>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>> share strage). Because
of the rate limitation of device
>>>>> (such as
>>>>>>>>>>>>> iops),
>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>> one VM has large disk
operation, it may affect the disk
>>>>>>>>>>> performance of
>>>>>>>>>>>>>>>>> other VMs running on
the same storage device.
>>>>>>>>>>>>>>>>> It is neccesary to set
the maximum rate and limit the disk
>>>>> I/O
>>>>>>> of
>>>>>>>>>>> VMs.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> The feature includes:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> (1) set the maximum rate
of VMs (in disk_offering, and
>> global
>>>>>>>>>>>>>>>>> configuration)
>>>>>>>>>>>>>>>>> (2) change the maximum
rate of VMs
>>>>>>>>>>>>>>>>> (3) limit the disk rate
(total bps and iops)
>>>>>>>>>>>>>>>>> JIRA ticket:
>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192
>>>>>>>>>>>>>>>>> FS (I will update later)
:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>> 
>>>>> 
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>>>>>>> Merge check list :-
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> * Did you check the branch's
RAT execution success?
>>>>>>>>>>>>>>>>> Yes
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> * Are there new dependencies
introduced?
>>>>>>>>>>>>>>>>> No
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> * What automated testing
(unit and integration) is included
>>>>> in
>>>>>>>>> the
>>>>>>>>>>> new
>>>>>>>>>>>>>>>>> feature?
>>>>>>>>>>>>>>>>> Unit tests are added.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> * What testing has been
done to check for potential
>>>>> regressions?
>>>>>>>>>>>>>>>>> (1) set the bytes rate
and IOPS rate on CloudStack UI.
>>>>>>>>>>>>>>>>> (2) VM operations, including
>>>>>>>>>>>>>>>>> deploy, stop, start,
reboot, destroy, expunge. migrate,
>>>>> restore
>>>>>>>>>>>>>>>>> (3) Volume operations,
including
>>>>>>>>>>>>>>>>> Attach, Detach
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> To review the code, you
can try
>>>>>>>>>>>>>>>>> git diff c30057635d04a2396f84c588127d7ebe42e503a7
>>>>>>>>>>>>>>>>> f2e5591b710d04cc86815044f5823e73a4a58944
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>>>>>> Wei
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>> 
>>>>> 
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
>>>>>>>>>>>>>>>>> [2] refs/heads/disk_io_throttling
>>>>>>>>>>>>>>>>> [3] https://issues.apache.org/jira/browse/CLOUDSTACK-1301<
>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071
>>>>>>>>>>>> (CLOUDSTACK-1301
>>>>>>>>>>>>> -
>>>>>>>>>>>>>>> VM Disk I/O Throttling)
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire
Inc.*
>>>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>>>> Advancing the way the world uses
the
>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>>>> *™*
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> --
>>>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>>>> o: 303.746.7302
>>>>>>>>>>>> Advancing the way the world uses the
>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>>>> *™*
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> --
>>>>>>>>>> *Mike Tutkowski*
>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>>> o: 303.746.7302
>>>>>>>>>> Advancing the way the world uses the cloud<
>>>>>>>>> http://solidfire.com/solution/overview/?video=play>
>>>>>>>>>> *™*
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> --
>>>>>>>>> *Mike Tutkowski*
>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>>>> o: 303.746.7302
>>>>>>>>> Advancing the way the world uses the
>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>>>>> *™*
>>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> --
>>>>>> *Mike Tutkowski*
>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>> e: mike.tutkowski@solidfire.com
>>>>>> o: 303.746.7302
>>>>>> Advancing the way the world uses the
>>>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>>>> *™*
>>>>> 
>>>>> 
>>>> 
>>>> 
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkowski@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the cloud<
>> http://solidfire.com/solution/overview/?video=play>
>>>> *™*
>>>> 
>>> 
>>> 
>>> 
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the
>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> *™*
>> 
>> 
> 
> 
> -- 
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *™*


Mime
View raw message