incubator-cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tamas Monos <tam...@veber.co.uk>
Subject RE: Vmware CPU Cap
Date Wed, 23 May 2012 22:19:46 GMT
I'm afraid I don't follow. 
I was talking about multiple core VMs and not single core ones.
I said if I have a 4 core VM with 3Ghz core speed and I run 4 cpuburn threads I could get
approximately the performance of a single core at 12Ghz.

-----Original Message-----
From: Edison Su [mailto:Edison.su@citrix.com] 
Sent: 23 May 2012 23:06
To: cloudstack-dev@incubator.apache.org
Subject: RE: Vmware CPU Cap

Single core means there is only one scheduling entity, that hypervisor can schedule for the
VM, so there is no parallel processing at all. Hypervisor can not schedule a single core VM
to multiple physical cpu cores at the same time.


> -----Original Message-----
> From: Tamas Monos [mailto:tamasm@veber.co.uk]
> Sent: Wednesday, May 23, 2012 2:59 PM
> To: cloudstack-dev@incubator.apache.org
> Subject: RE: Vmware CPU Cap
>
> Hi,
>
> Before we would touch that rule 2:
> While utilizing 12Ghz on a single core is not possible it is possible 
> to have effective 12Ghz on 4 cores at 3Ghz via parallel processing.
> If I start 4 cpuburn threads on a quad core it will use all four 3Ghz 
> cores flat out "giving similar speed as 1 core@12Ghz would be"*. As 
> vmware uses the limit parameter as limit to the whole CPU utilization 
> rule 2 approach would prevent me effectively utilizing the CPU on 
> vmware because if we say the CPU speed in Mhz cannot be larger than 
> the core speed I would be limited to an effective speed of core- 
> speed/number_of_cores. This would mean  3000/4=750Mhz per core on a 
> 4vCPU VM. If you set "core_number*core_speed" as limit you already 
> broke rule 2 as the CPU is set higher than its core speed.
>
> I believe it good as it is now.
>
> Regards
>
> -----Original Message-----
> From: Prachi Damle [mailto:Prachi.Damle@citrix.com]
> Sent: 23 May 2012 22:36
> To: cloudstack-dev@incubator.apache.org
> Subject: RE: Vmware CPU Cap
>
> Yeah, rule#1 and #3 are considered by the current Host Allocator;
> rule#2 to compare just the speed is missing.
>
> Prachi
>
> -----Original Message-----
> From: Anthony Xu [mailto:Xuefei.Xu@citrix.com]
> Sent: Wednesday, May 23, 2012 2:06 PM
> To: cloudstack-dev@incubator.apache.org
> Subject: RE: Vmware CPU Cap
>
> Checked the code, I think we miss rule 2 in allocator.
>
> Anthony
>
> > -----Original Message-----
> > From: Anthony Xu [mailto:Xuefei.Xu@citrix.com]
> > Sent: Wednesday, May 23, 2012 1:50 PM
> > To: cloudstack-dev@incubator.apache.org
> > Subject: RE: Vmware CPU Cap
> >
> > I'm pretty sure we have that before we redesign allocator.
> >
> > Anthony
> >
> > > -----Original Message-----
> > > From: Alex Huang [mailto:Alex.Huang@citrix.com]
> > > Sent: Wednesday, May 23, 2012 1:44 PM
> > > To: cloudstack-dev@incubator.apache.org
> > > Subject: RE: Vmware CPU Cap
> > >
> > > I'm surprised.  I definitely recall having that conversation 
> > > before about never exceeding the physical limits of the 
> > > hypervisor.  I'm pretty sure we have test cases against that as well.
> > >
> > > --Alex
> > >
> > > > -----Original Message-----
> > > > From: Edison Su [mailto:Edison.su@citrix.com]
> > > > Sent: Wednesday, May 23, 2012 1:39 PM
> > > > To: cloudstack-dev@incubator.apache.org
> > > > Subject: RE: Vmware CPU Cap
> > > >
> > > > I think it's a general issue for all the hypervisors we supported.
> > > >
> > > > > -----Original Message-----
> > > > > From: Alex Huang [mailto:Alex.Huang@citrix.com]
> > > > > Sent: Wednesday, May 23, 2012 1:32 PM
> > > > > To: cloudstack-dev@incubator.apache.org
> > > > > Subject: RE: Vmware CPU Cap
> > > > >
> > > > > That's because no one wrote a host allocator for VmWare.
> > > > >
> > > > > The relationship between DeploymentPlanner and HostAllocator
> and
> > > > > StoragePoolAllocator is as follows:
> > > > >
> > > > > DeploymentPlanner deals with heuristics and affinity rules of
> > > placing
> > > > > a VM.  Once it determines a set of hosts that matches, it then
> > asks
> > > > > the HostAllocator to work on the limitations of the type of
> > > hypervisor.
> > > > > There was never a HostAllocator written for VmWare.  It just 
> > > > > uses
> > > the
> > > > > one that was written originally for XenServer.  Hence the bug.
> > > > >
> > > > > There's similar differentiation for DeploymentPlanner and 
> > > > > StoragePoolAllocator and similar bugs exists, especially if
> > someone
> > > > > adds a completely new type of storage pool but decides to just
> > > reuse
> > > > > the current set of StoragePoolAllocators.
> > > > >
> > > > > --Alex
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Edison Su [mailto:Edison.su@citrix.com]
> > > > > > Sent: Wednesday, May 23, 2012 1:24 PM
> > > > > > To: cloudstack-dev@incubator.apache.org; 'tamasm@veber.co.uk'
> > > > > > Subject: RE: Vmware CPU Cap
> > > > > >
> > > > > > Thanks for your input, I find another bug in cloudstack...
> > > > > > Let's dive little bit deeper into how the CPU allocation
> works.
> > > > > > VMware(the same for other hypervisors(Xen/KVM)) uses
> > proportional
> > > > > share
> > > > > > based scheduling algorithm([1],[2]). It means "If a virtual
> > > machine
> > > > > has twice
> > > > > > as many shares of a resource as another virtual machine, it
> is
> > > > > entitled to
> > > > > > consume twice as much of that resource when these two 
> > > > > > virtual
> > > > > machines
> > > > > > are competing for resources."
> > > > > > There are two questions:
> > > > > > 1. Where is the share coming from? In Cloudstack, the share
> is
> > > > > calculated
> > > > > > from (CPU MHz * CPU cores).
> > > > > > 2. How the share of a VM is mapped to the physical CPU by 
> > > > > > the
> > > > > hypervisor?
> > > > > > Of cause, it depends on the hypervisor implementation, but
> > there
> > > are
> > > > > some
> > > > > > general rules that we need to follow, as we are living in 
> > > > > > the
> > > real
> > > > > world:)
> > > > > >    Rule 1: num of cpu core of a VM should be <= num of cpu

> > > > > > core
> > > of
> > > > > the
> > > > > > hypervisor host. It doesn't make sense to running a 12 core
> VM
> > on
> > > a
> > > > > single
> > > > > > core host, even some hypervisors, such as KVM, can do that.
> > > > > Hypervisor will
> > > > > > be busy at scheduling VCPU(the VCPU context switch is much
> > > heavier
> > > > > than
> > > > > > process context switch), thus bad performance for the VM.
> > > > > >    Rule 2: The frequency of a VCPU should be <= the 
> > > > > > frequency
> > of
> > > the
> > > > > host
> > > > > > hypervisor host. Can you run an one core * 12Ghz VM on a 
> > > > > > 3Ghz
> > > > > > *
> > 4
> > > > > core
> > > > > > physical CPU? Nope, in an unit time, the max freq of VCPU 
> > > > > > can
> > get
> > > is
> > > > > the max
> > > > > > freq of physical cpu core, that's the physical LAW.
> > > > > >    Rule 3: the total share of VMs <= total share of host
> > > > > hypervisor(in case of
> > > > > > no CPU overcommit).
> > > > > >
> > > > > > Currently, in cloudstack host allocator, only rule 3 is 
> > > > > > taken
> > > into
> > > > > consideration.
> > > > > >
> > > > > > [1] www.cs.uiuc.edu/class/sp10/cs423/lectures/17-VMM-
> > resource.pdf
> > > > > > [2]
> > > > > >
> > > > http://www.vmware.com/pdf/vsphere4/r40/vsp_40_resource_mgmt.pdf
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Tamas Monos [mailto:tamasm@veber.co.uk]
> > > > > > > Sent: Tuesday, May 22, 2012 5:41 PM
> > > > > > > To: cloudstack-dev@incubator.apache.org
> > > > > > > Subject: RE: Vmware CPU Cap
> > > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > I still don't think this is an issue as the CPU Mhz limit

> > > > > > > and
> > > the
> > > > > > > number of cores are independent.
> > > > > > > CPU manufacturers sell 2,4,6 cores at 3Ghz and not 
> > > > > > > 6,12,18Ghz
> > > CPUs.
> > > > > > >
> > > > > > > So I think it is good how it works but the
> > > "number_of_cores*Mhz"
> > > > > while
> > > > > > > allocating should not multiply so that is the bug :) What
> > > vmware
> > > > > does
> > > > > > > with multiplying the cores with the core speed is bad as
I
> > > can't
> > > > > have
> > > > > > > a 1vCPU VM at 12Ghz on a 3Ghz quad core if you know where

> > > > > > > I'm
> > > > > coming
> > > > > > > from....
> > > > > > >
> > > > > > > Regards
> > > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Tamas Monos [mailto:tamasm@veber.co.uk]
> > > > > > > Sent: 23 May 2012 01:27
> > > > > > > To: cloudstack-dev@incubator.apache.org
> > > > > > > Subject: RE: Vmware CPU Cap
> > > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > I have confused myself too because if I have a look the
> > > database
> > > > > and
> > > > > > > dump the service_offerings table the limit for me is set

> > > > > > > to
> > > > > > > 2000,3000,4000 and not 1000.
> > > > > > > So a Mhz limit set to 4000 and 4 cores will end up as a
> quad
> > > core
> > > > > box
> > > > > > > at 4Ghz. I remember now I had to set CPU overprovisioning
> to
> > 4
> > > as
> > > > > > > CloudStack took away 4x4Ghz=16Ghz of the available CPU.
> > > > > > > So Cloudstack sets the VM CPU Mhz limit what the actual

> > > > > > > limit
> > > is
> > > > > set
> > > > > > > to in the offering but does not multiply the Mhz limit
by 
> > > > > > > the
> > > > > number
> > > > > > > of cores when setting the limit on vmware. However takes

> > > > > > > away "number_of_cores*Mhz limit" from the available CPU

> > > > > > > capacity
> > > when
> > > > > > > allocating.
> > > > > > >
> > > > > > > I'm getting confused myself so I'm not sure if this is
bug
> > now
> > > or
> > > > > not
> > > > > > > either :) I'm using 3.0.3
> > > > > > >
> > > > > > > Regards
> > > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Edison Su [mailto:Edison.su@citrix.com]
> > > > > > > Sent: 23 May 2012 00:32
> > > > > > > To: cloudstack-dev@incubator.apache.org; Tamas Monos
> > > > > > > Subject: RE: Vmware CPU Cap
> > > > > > >
> > > > > > > But in Diego'case, the limit is set as 1000Mhz, while his
> > > service
> > > > > > > offering is 1000Mhz * 2 cores.
> > > > > > > Which version of cloudstack are you using? Maybe it's a
> > > regression.
> > > > > > >
> > > > > > > > -----Original Message-----
> > > > > > > > From: Tamas Monos [mailto:tamasm@veber.co.uk]
> > > > > > > > Sent: Tuesday, May 22, 2012 4:26 PM
> > > > > > > > To: cloudstack-dev@incubator.apache.org
> > > > > > > > Subject: RE: Vmware CPU Cap
> > > > > > > >
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > I have a service offering with 1000Mhz limit and 4
cpu
> > cores.
> > > > > That
> > > > > > > > sums up to 4000Mhz.
> > > > > > > > Cloudstack sets the limit to 4Ghz on the virtual machine
> > and
> > > > > > > > when
> > > > > I
> > > > > > > > put load on it vmware balances the load between the
4 
> > > > > > > > cores
> > > > > allowing
> > > > > > > > them to use 1000Mhz each.
> > > > > > > > I do not see any bugs here.
> > > > > > > >
> > > > > > > > Apologies if the "meant per CPU core" was incorrect.

> > > > > > > > What I meant
> > > > > is
> > > > > > > > described above.
> > > > > > > >
> > > > > > > > Regards
> > > > > > > >
> > > > > > > > -----Original Message-----
> > > > > > > > From: Edison Su [mailto:Edison.su@citrix.com]
> > > > > > > > Sent: 23 May 2012 00:05
> > > > > > > > To: cloudstack-dev@incubator.apache.org
> > > > > > > > Subject: RE: Vmware CPU Cap
> > > > > > > >
> > > > > > > > Nope, from the document, the limit is set on whole
vm:
> > > > > > > > CPU Limits
> > > > > > > >
> > > > > > > > When a CPU Limit is set on a virtual machine resource
> > > settings,
> > > > > the
> > > > > > > > virtual machine is deliberately held from being 
> > > > > > > > scheduled
> > to
> > > a
> > > > > PCPU
> > > > > > > > when it has used up its allocated CPU resource. This
> > happens
> > > > > > > > regardless of the CPU utilization. If the limit is
set 
> > > > > > > > to 500MHz, the virtual machine is descheduled from
the 
> > > > > > > > PCPU
> > and
> > > has
> > > > > > > > to wait before
> > > > > > > it
> > > > > > > > is allowed to be scheduled again. As such, the virtual
> > > machine
> > > > > might
> > > > > > > > experience performance degradation.
> > > > > > > >
> > > > > > > > Note: For an SMP virtual machine, the sum of all vCPUs
> > cannot
> > > > > exceed
> > > > > > > > the specified limit. For example, 4 vCPU virtual machine
> > with
> > > a
> > > > > > > > limit of 1200MHz and equal load among vCPUs would
result 
> > > > > > > > in
> > a
> > > > > > > > max
> > > > > of
> > > > > > > > 300MHz per vCPU.
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> > http://kb.vmware.com/selfservice/documentLinkInt.do?micrositeID=&pop
> > > > > > u
> > > > > > p
> > > > > > > > =
> > > > > > > > true&languageId=&externalID=1033115
> > > > > > > >
> > > > > > > > > -----Original Message-----
> > > > > > > > > From: Tamas Monos [mailto:tamasm@veber.co.uk]
> > > > > > > > > Sent: Tuesday, May 22, 2012 3:43 PM
> > > > > > > > > To: cloudstack-dev@incubator.apache.org
> > > > > > > > > Subject: RE: Vmware CPU Cap
> > > > > > > > >
> > > > > > > > > Hi,
> > > > > > > > >
> > > > > > > > > No I don't think this is a bug. When you set
1000Mhz 
> > > > > > > > > as
> > CPU
> > > > > > > > > cap
> > > > > > > that
> > > > > > > > > is meant per core. vmWare will limit each CPU
core to
> > > 1000Mhz.
> > > > > > > > > As you gave 2 CPU cores that is 2000Mhz effective.

> > > > > > > > > That
> > is
> > > how
> > > > > > > > > vmware works.
> > > > > > > > >
> > > > > > > > > I have setup my offerings all to 1000Mhz as speed
and
> > just
> > > > > > > > > increasing the number of cores.
> > > > > > > > > 1 core ends up being 1x1000Mhz
> > > > > > > > > 2 core ends up being 2x1000Mhz=2000Mhz ...
> > > > > > > > > ...
> > > > > > > > >
> > > > > > > > > I'm actually using it in production and works
quite
> well
> > as
> > > > > > > > Cloudstack
> > > > > > > > > "allocates" 4000Mhz when I'm using 4x1000Mhz
cores and
> > > vmware
> > > > > > > > cleverly
> > > > > > > > > balances between the cores as you put load on
it and
> not
> > > > > letting
> > > > > > > any
> > > > > > > > > of the cores above the set limit of 1000Mhz.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Regards
> > > > > > > > >
> > > > > > > > > -----Original Message-----
> > > > > > > > > From: Diego Spinola Castro
> > [mailto:spinolacastro@gmail.com]
> > > > > > > > > Sent: 22 May 2012 19:34
> > > > > > > > > To: cloudstack-dev@incubator.apache.org
> > > > > > > > > Subject: Re: Vmware CPU Cap
> > > > > > > > >
> > > > > > > > > I forgot the link:
> > > > > > > > > http://imageshack.us/photo/my-images/848/cpulimit.png/
> > > > > > > > >
> > > > > > > > > 2012/5/22 Diego Spinola Castro
> <spinolacastro@gmail.com>
> > > > > > > > >
> > > > > > > > > > I believe that is a bug with cpu cap and
vmware.
> > > > > > > > > >
> > > > > > > > > > To reproduce:
> > > > > > > > > >
> > > > > > > > > > Create a offering with 2 cores and 1000mhz.
> > > > > > > > > > Enable CPU CAP.
> > > > > > > > > >
> > > > > > > > > > After created instance , cs create a vm
with 2 cores
> > and
> > > > > 1000mhz
> > > > > > > > > > of
> > > > > > > > > limit.
> > > > > > > > > >
> > > > > > > > > > I don't know for sure if is a bug, but vmware
gives
> > > 1000mhz
> > > > > > > shared
> > > > > > > > > > with cores.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Diego
> > > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
>
>




Mime
View raw message