cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wei ZHOU <ustcweiz...@gmail.com>
Subject Re: [MERGE] disk_io_throttling to MASTER
Date Mon, 03 Jun 2013 06:13:21 GMT
Hi John, Mike

I hope Mike's aswer helps you. I am trying to adding more.

(1) I think billing should depend on IO statistics rather than IOPS
limitation. Please review disk_io_stat if you have time.   disk_io_stat can
get the IO statistics including bytes/iops read/write for an individual
virtual machine.

(2) Do you mean IOPS runtime change? KVM supports setting IOPS/BPS
limitation for a running virtual machine through command line. However,
CloudStack does not support changing the parameters of a created offering
(computer offering or disk offering).

(3) It is a good question. Maybe it is better to commit Mike's patch after
disk_io_throttling as Mike needs to consider the limitation in hypervisor
type, I think.

- Wei


2013/6/3 John Burwell <jburwell@basho.com>

> Mike,
>
> The things I want to understand are the following:
>
>    1) Is there value in capturing IOPS policies be captured in a common
> data model (e.g. for billing/usage purposes, expressing offerings).
>     2) Should there be a common interface model for reasoning about IOP
> provisioning at runtime?
>     3) How are conflicting provisioned IOPS configurations between a
> hypervisor and storage device reconciled?  In particular, a scenario where
> is lead to believe (and billed) for more IOPS configured for a VM than a
> storage device has been configured to deliver.  Another scenario could a
> consistent configuration between a VM and a storage device at creation
> time, but a later modification to storage device introduces logical
> inconsistency.
>
> Thanks,
> -John
>
> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski <mike.tutkowski@solidfire.com>
> wrote:
>
> Hi John,
>
> I believe Wei's feature deals with controlling the max number of IOPS from
> the hypervisor side.
>
> My feature is focused on controlling IOPS from the storage system side.
>
> I hope that helps. :)
>
>
> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <jburwell@basho.com> wrote:
>
> > Wei,
> >
> > My opinion is that no features should be merged until all functional
> > issues have been resolved and it is ready to turn over to test.  Until
> the
> > total Ops vs discrete read/write ops issue is addressed and re-reviewed
> by
> > Wido, I don't think this criteria has been satisfied.
> >
> > Also, how does this work intersect/compliment the SolidFire patch (
> > https://reviews.apache.org/r/11479/)?  As I understand it that work is
> > also involves provisioned IOPS.  I would like to ensure we don't have a
> > scenario where provisioned IOPS in KVM and SolidFire are unnecessarily
> > incompatible.
> >
> > Thanks,
> > -John
> >
> > On Jun 1, 2013, at 6:47 AM, Wei ZHOU <ustcweizhou@gmail.com> wrote:
> >
> > Wido,
> >
> >
> > Sure. I will change it next week.
> >
> >
> > -Wei
> >
> >
> >
> > 2013/6/1 Wido den Hollander <wido@widodh.nl>
> >
> >
> > Hi Wei,
> >
> >
> >
> > On 06/01/2013 08:24 AM, Wei ZHOU wrote:
> >
> >
> > Wido,
> >
> >
> > Exactly. I have pushed the features into master.
> >
> >
> > If anyone object thems for technical reason till Monday, I will revert
> >
> > them.
> >
> >
> > For the sake of clarity I just want to mention again that we should
> change
> >
> > the total IOps to R/W IOps asap so that we never release a version with
> >
> > only total IOps.
> >
> >
> > You laid the groundwork for the I/O throttling and that's great! We
> should
> >
> > however prevent that we create legacy from day #1.
> >
> >
> > Wido
> >
> >
> > -Wei
> >
> >
> >
> > 2013/5/31 Wido den Hollander <wido@widodh.nl>
> >
> >
> > On 05/31/2013 03:59 PM, John Burwell wrote:
> >
> >
> > Wido,
> >
> >
> > +1 -- this enhancement must to discretely support read and write IOPS.
> >
> > I
> >
> > don't see how it could be fixed later because I don't see how we
> >
> > correctly
> >
> > split total IOPS into read and write.  Therefore, we would be stuck
> >
> > with a
> >
> > total unless/until we decided to break backwards compatibility.
> >
> >
> >
> > What Wei meant was merging it into master now so that it will go in the
> >
> > 4.2 branch and add Read / Write IOps before the 4.2 release so that 4.2
> >
> > will be released with Read and Write instead of Total IOps.
> >
> >
> > This is to make the May 31st feature freeze date. But if the window moves
> >
> > (see other threads) then it won't be necessary to do that.
> >
> >
> > Wido
> >
> >
> >
> >  I also completely agree that there is no association between network
> >
> > and
> >
> >
> > disk I/O.
> >
> >
> > Thanks,
> >
> > -John
> >
> >
> > On May 31, 2013, at 9:51 AM, Wido den Hollander <wido@widodh.nl> wrote:
> >
> >
> >  Hi Wei,
> >
> >
> >
> > On 05/31/2013 03:13 PM, Wei ZHOU wrote:
> >
> >
> > Hi Wido,
> >
> >
> > Thanks. Good question.
> >
> >
> > I  thought about at the beginning. Finally I decided to ignore the
> >
> > difference of read and write mainly because the network throttling did
> >
> > not
> >
> > care the difference of sent and received bytes as well.
> >
> > That reasoning seems odd. Networking and disk I/O completely different.
> >
> >
> > Disk I/O is much more expensive in most situations then network
> >
> > bandwith.
> >
> >
> >  Implementing it will be some copy-paste work. It could be
> >
> > implemented in
> >
> >
> > few days. For the deadline of feature freeze, I will implement it
> >
> > after
> >
> >  that , if needed.
> >
> >
> >
> > It think it's a feature we can't miss. But if it goes into the 4.2
> >
> >  window we have to make sure we don't release with only total IOps and
> >
> > fix
> >
> > it in 4.3, that will confuse users.
> >
> >
> > Wido
> >
> >
> >  -Wei
> >
> >
> >
> >
> >
> >
> > 2013/5/31 Wido den Hollander <wido@widodh.nl>
> >
> >
> >  Hi Wei,
> >
> >
> >
> >
> > On 05/30/2013 06:03 PM, Wei ZHOU wrote:
> >
> >
> >  Hi,
> >
> >
> >  I would like to merge disk_io_throttling branch into master.
> >
> >  If nobody object, I will merge into master in 48 hours.
> >
> > The purpose is :
> >
> >
> > Virtual machines are running on the same storage device (local
> >
> > storage or
> >
> >  share strage). Because of the rate limitation of device (such as
> >
> > iops), if
> >
> >  one VM has large disk operation, it may affect the disk performance
> >
> >  of
> >
> > other VMs running on the same storage device.
> >
> > It is neccesary to set the maximum rate and limit the disk I/O of
> >
> >  VMs.
> >
> >
> >
> >  Looking at the code I see you make no difference between Read and
> >
> > Write
> >
> > IOps.
> >
> >
> > Qemu and libvirt support setting both a different rate for Read and
> >
> >  Write
> >
> > IOps which could benefit a lot of users.
> >
> >
> >  It's also strange, in the polling side you collect both the Read and
> >
> >  Write
> >
> > IOps, but on the throttling side you only go for a global value.
> >
> >
> > Write IOps are usually much more expensive then Read IOps, so it
> >
> > seems
> >
> > like a valid use-case where that an admin would set a lower value for
> >
> >  write
> >
> > IOps vs Read IOps.
> >
> >
> >  Since this only supports KVM at this point I think it would be of
> >
> > great
> >
> > value to at least have the mechanism in place to support both,
> >
> > implementing
> >
> > this later would be a lot of work.
> >
> >
> >  If a hypervisor doesn't support setting different values for read and
> >
> > write you can always sum both up and set that as the total limit.
> >
> >
> > Can you explain why you implemented it this way?
> >
> >
> > Wido
> >
> >
> >   The feature includes:
> >
> >
> >
> > (1) set the maximum rate of VMs (in disk_offering, and global
> >
> >  configuration)
> >
> >  (2) change the maximum rate of VMs
> >
> > (3) limit the disk rate (total bps and iops)
> >
> >  JIRA ticket: https://issues.apache.org/****
> >
> >  jira/browse/CLOUDSTACK-1192<ht**tps://issues.apache.org/****
> >
> >  jira/browse/CLOUDSTACK-1192<
> > https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>
> >
> >  <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1192<
> > http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>
> >
> > <**
> > https://issues.apache.org/**jira/browse/CLOUDSTACK-1192<
> https://issues.apache.org/jira/browse/CLOUDSTACK-1192>
> >
> >
> >
> >  FS (I will update later) :
> >
> >
> > https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> >
> > <
> > https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<
> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
> >
> > VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****<
> > http://cwiki.apache.org/confluence/**>
> >
> > display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
> >
> >  apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling<
> >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> > >
> >
> >
> >
> >  Merge check list :-
> >
> >
> > * Did you check the branch's RAT execution success?
> >
> > Yes
> >
> >
> >  * Are there new dependencies introduced?
> >
> >  No
> >
> >
> >  * What automated testing (unit and integration) is included in the
> >
> > new
> >
> > feature?
> >
> >  Unit tests are added.
> >
> >
> >  * What testing has been done to check for potential regressions?
> >
> > (1) set the bytes rate and IOPS rate on CloudStack UI.
> >
> > (2) VM operations, including
> >
> >  deploy, stop, start, reboot, destroy, expunge. migrate, restore
> >
> > (3) Volume operations, including
> >
> > Attach, Detach
> >
> >
> > To review the code, you can try
> >
> >   git diff c30057635d04a2396f84c588127d7e******be42e503a7
> >
> > f2e5591b710d04cc86815044f5823e******73a4a58944
> >
> >
> > Best regards,
> >
> > Wei
> >
> >
> >  [1]
> >
> >
> > https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******<
> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****>
> >
> > <
> > https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**<
> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**>
> >
> > VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****<
> > http://cwiki.apache.org/confluence/**>
> >
> > display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.**
> >
> >  apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling<
> >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling
> > >
> >
> >
> >
> > [2] refs/heads/disk_io_throttling
> >
> >  [3]
> > https://issues.apache.org/******jira/browse/CLOUDSTACK-1301<
> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301>
> >
> > <ht**tps://issues.apache.org/****jira/browse/CLOUDSTACK-1301<
> > https://issues.apache.org/**jira/browse/CLOUDSTACK-1301>
> >
> > <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1301<
> > http://issues.apache.org/jira/**browse/CLOUDSTACK-1301>
> >
> > <**
> > https://issues.apache.org/**jira/browse/CLOUDSTACK-1301<
> https://issues.apache.org/jira/browse/CLOUDSTACK-1301>
> >
> >
> >
> > <ht**tps://issues.apache.org/****jira/**browse/CLOUDSTACK-2071<
> > http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071>
> >
> > **<
> > http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071<
> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071>
> >
> > <**
> > https://issues.apache.org/****jira/browse/CLOUDSTACK-2071<
> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071>
> >
> > <h**ttps://issues.apache.org/jira/**browse/CLOUDSTACK-2071<
> > https://issues.apache.org/jira/browse/CLOUDSTACK-2071>
> >
> >
> >
> >  (**CLOUDSTACK-1301
> >
> >  -     VM Disk I/O Throttling)
> >
> >
> >
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *™*
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message