cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrija Panic <andrija.pa...@gmail.com>
Subject Re: KVM CPU Overprovisioning issues.
Date Fri, 17 Apr 2015 21:10:02 GMT
If not mistaken, there is global overprovisioning factor, and there is
cluster level overprovisioning - the last one takes presedence I think...

On 17 April 2015 at 23:01, Grayson Head <grayson@graysonhead.net> wrote:

> I'm trying to figure out how CPU over-provisioning works in
> cloudstack. It seems that no matter how much I change the
> over-provisioning factor value in global settings, I cannot
> over-provision CPU allocation on the hosts. I suspect that I'm just
> having a lack of understanding on my part, but it seems like I should
> be able to increase the number of VMs I can deploy by increasing the
> CPU over-provisioning value (given that I'm not close to any other
> resource limits). But this doesn't seem to be the case in practice.
>
> https://screencloud.net/v/l1X7
>
> https://screencloud.net/v/9ifx
>
> Error message when trying to create additional machine via API.
>
> grayson@gworkstation:~$ cloudmonkey deploy virtualmachine
> zoneid=a4c20770-f17e-425e-bb21-413dd610b975
> serviceofferingid=4b391121-49c7-4434-883d-ffe569f63a3b
> templateid=e5701f73-b0c3-4dd7-8b2d-b8bf78875a85 keypair=key1
> networkids=86e6ded0-6155-449b-a4bd-03411bf0a9cd name=tvmsman1
> displayname &
> [1] 3907
> Async job 09148c21-3c42-4334-93b2-428b48fb1fa2 failed
> Error 533, Unable to create a deployment for VM[User|i-15-1151-VM]
>  accountid = a4e6eaed-9b06-4806-98f9-4fcc781b773f
> cmd = org.apache.cloudstack.api.command.user.vm.DeployVMCmd
> created = 2015-04-17T13:46:22-0700
> jobid = 09148c21-3c42-4334-93b2-428b48fb1fa2
> jobprocstatus = 0
> jobresult:
> errorcode = 533
> errortext = Unable to create a deployment for VM[User|i-15-1151-VM]
> jobresultcode = 530
> jobresulttype = object
> jobstatus = 2
> userid = c7a0889e-fcec-4095-9d07-d29ea3c1e5d9
>
>
> (local) mycloudmonkey> list hosts
>
> filter=name,clustername,cpuallocated,cpunumber,cpusockets,cpuspeed,cpuused,cpuwithoverprovisioning,
> name = v-78-VM
>
> ================================================================================
> name = s-871-VM
>
> ================================================================================
> name = cl2hyper2
> clustername = CL2
> cpuallocated = 100.3%
> cpunumber = 8
> cpusockets = 2
> cpuspeed = 1994
> cpuused = 1.2%
> cpuwithoverprovisioning = 63808.0
>
> ================================================================================
> name = v-1-VM
>
> ================================================================================
> name = cl2hyper1
> clustername = CL2
> cpuallocated = 81.45%
> cpunumber = 8
> cpusockets = 2
> cpuspeed = 1995
> cpuused = 0.6%
> cpuwithoverprovisioning = 63840.0
>
> ================================================================================
>
>
>
> 2015-04-17 13:46:22,517 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (catalina-exec-25:ctx-7b0eb43a ctx-751b2eca ctx-5c8a75fc) submit async
> job-7312, details: AsyncJobVO {id:7312, userId: 15, accountId: 15,
> instanceType: VirtualMachine, instanceId: 1151, cmd:
> org.apache.cloudstack.api.command.user.vm.DeployVMCmd, cmdInfo:
>
> {"serviceofferingid":"4b391121-49c7-4434-883d-ffe569f63a3b","cmdEventType":"VM.CREATE","ctxUserId":"15","zoneid":"a4c20770-f17e-425e-bb21-413dd610b975","signatureversion":"3","httpmethod":"GET","templateid":"e5701f73-b0c3-4dd7-8b2d-b8bf78875a85","networkids":"86e6ded0-6155-449b-a4bd-03411bf0a9cd","apiKey":"wrxDdS8yi2kJPe8UrNdOdLtp55mCSKskk3MyTJ1ucIApdMoYnOWsd-dmigPywD1KC-6BVfVmKiJVenEM_kovpw","id":"1151","response":"json","ctxDetails":"{\"com.cloud.template.VirtualMachineTemplate\":\"e5701f73-b0c3-4dd7-8b2d-b8bf78875a85\",\"com.cloud.vm.VirtualMachine\":\"6b3ace85-8e4c-4467-afdf-b511136ca6b1\",\"com.cloud.network.Network\":\"86e6ded0-6155-449b-a4bd-03411bf0a9cd\",\"com.cloud.dc.DataCenter\":\"a4c20770-f17e-425e-bb21-413dd610b975\",\"com.cloud.offering.ServiceOffering\":\"4b391121-49c7-4434-883d-ffe569f63a3b\"}","expires":"2015-04-17T20:56:22+0000","keypair":"key1","name":"tvmsman1","uuid":"6b3ace85-8e4c-4467-afdf-b511136ca6b1","ctxAccountId":"15","ctxStartEventId":"11922","signature":"8Ebg1+qjfnk5nv7xz1GtdSju2ws\u003d"},
> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
> result: null, initMsid: 345050015385, completeMsid: null, lastUpdated:
> null, lastPolled: null, created: null}
> 2015-04-17 13:46:22,518 INFO  [o.a.c.f.j.i.AsyncJobMonitor]
> (API-Job-Executor-16:ctx-e039eccb job-7312) Add job-7312 into job
> monitoring
> 2015-04-17 13:46:22,518 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-16:ctx-e039eccb job-7312) Executing AsyncJobVO
> {id:7312, userId: 15, accountId: 15, instanceType: VirtualMachine,
> instanceId: 1151, cmd:
> org.apache.cloudstack.api.command.user.vm.DeployVMCmd, cmdInfo:
>
> {"serviceofferingid":"4b391121-49c7-4434-883d-ffe569f63a3b","cmdEventType":"VM.CREATE","ctxUserId":"15","zoneid":"a4c20770-f17e-425e-bb21-413dd610b975","signatureversion":"3","httpmethod":"GET","templateid":"e5701f73-b0c3-4dd7-8b2d-b8bf78875a85","networkids":"86e6ded0-6155-449b-a4bd-03411bf0a9cd","apiKey":"wrxDdS8yi2kJPe8UrNdOdLtp55mCSKskk3MyTJ1ucIApdMoYnOWsd-dmigPywD1KC-6BVfVmKiJVenEM_kovpw","id":"1151","response":"json","ctxDetails":"{\"com.cloud.template.VirtualMachineTemplate\":\"e5701f73-b0c3-4dd7-8b2d-b8bf78875a85\",\"com.cloud.vm.VirtualMachine\":\"6b3ace85-8e4c-4467-afdf-b511136ca6b1\",\"com.cloud.network.Network\":\"86e6ded0-6155-449b-a4bd-03411bf0a9cd\",\"com.cloud.dc.DataCenter\":\"a4c20770-f17e-425e-bb21-413dd610b975\",\"com.cloud.offering.ServiceOffering\":\"4b391121-49c7-4434-883d-ffe569f63a3b\"}","expires":"2015-04-17T20:56:22+0000","keypair":"key1","name":"tvmsman1","uuid":"6b3ace85-8e4c-4467-afdf-b511136ca6b1","ctxAccountId":"15","ctxStartEventId":"11922","signature":"8Ebg1+qjfnk5nv7xz1GtdSju2ws\u003d"},
> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
> result: null, initMsid: 345050015385, completeMsid: null, lastUpdated:
> null, lastPolled: null, created: null}
> 2015-04-17 13:46:22,519 DEBUG [c.c.a.ApiServlet]
> (catalina-exec-25:ctx-7b0eb43a ctx-751b2eca ctx-5c8a75fc) ===END===
> 10.101.1.28 -- GET
>
> networkids=86e6ded0-6155-449b-a4bd-03411bf0a9cd&apiKey=wrxDdS8yi2kJPe8UrNdOdLtp55mCSKskk3MyTJ1ucIApdMoYnOWsd-dmigPywD1KC-6BVfVmKiJVenEM_kovpw&name=tvmsman1&serviceofferingid=4b391121-49c7-4434-883d-ffe569f63a3b&expires=2015-04-17T20%3A56%3A22%2B0000&signatureversion=3&zoneid=a4c20770-f17e-425e-bb21-413dd610b975&keypair=key1&command=deployVirtualMachine&templateid=e5701f73-b0c3-4dd7-8b2d-b8bf78875a85&signature=8Ebg1%2Bqjfnk5nv7xz1GtdSju2ws%3D&response=json
> 2015-04-17 13:46:22,522 DEBUG [o.a.c.a.BaseCmd]
> (API-Job-Executor-16:ctx-e039eccb job-7312 ctx-40bb38cb) Ignoring
> paremeter displayvm as the caller is not authorized to pass it in
> 2015-04-17 13:46:22,522 DEBUG [o.a.c.a.BaseCmd]
> (API-Job-Executor-16:ctx-e039eccb job-7312 ctx-40bb38cb) Ignoring
> paremeter deploymentplanner as the caller is not authorized to pass it
> in
> 2015-04-17 13:46:22,539 DEBUG [c.c.u.AccountManagerImpl]
> (API-Job-Executor-16:ctx-e039eccb job-7312 ctx-40bb38cb) Access to
> Acct[a4e6eaed-9b06-4806-98f9-4fcc781b773f-graysonp] granted to
> Acct[a4e6eaed-9b06-4806-98f9-4fcc781b773f-graysonp] by DomainChecker
> 2015-04-17 13:46:22,552 DEBUG [c.c.u.AccountManagerImpl]
> (API-Job-Executor-16:ctx-e039eccb job-7312 ctx-40bb38cb) Access to
> Acct[a4e6eaed-9b06-4806-98f9-4fcc781b773f-graysonp] granted to
> Acct[a4e6eaed-9b06-4806-98f9-4fcc781b773f-graysonp] by DomainChecker
> 2015-04-17 13:46:22,565 DEBUG [c.c.u.AccountManagerImpl]
> (API-Job-Executor-16:ctx-e039eccb job-7312 ctx-40bb38cb) Access to
> Tmpl[222-QCOW2-222-3-98769ef0-8ddf-300a-874f-0e9320ad02c9 granted to
> Acct[a4e6eaed-9b06-4806-98f9-4fcc781b773f-graysonp] by DomainChecker
> 2015-04-17 13:46:22,565 DEBUG [o.a.c.a.BaseCmd]
> (API-Job-Executor-16:ctx-e039eccb job-7312 ctx-40bb38cb) Ignoring
> paremeter displayvm as the caller is not authorized to pass it in
> 2015-04-17 13:46:22,565 DEBUG [o.a.c.a.BaseCmd]
> (API-Job-Executor-16:ctx-e039eccb job-7312 ctx-40bb38cb) Ignoring
> paremeter deploymentplanner as the caller is not authorized to pass it
> in
> 2015-04-17 13:46:22,566 WARN  [c.c.a.d.ParamGenericValidationWorker]
> (API-Job-Executor-16:ctx-e039eccb job-7312 ctx-40bb38cb) Received
> unknown parameters for command deployVirtualMachine. Unknown
> parameters : signatureversion expires
> 2015-04-17 13:46:22,588 DEBUG [c.c.u.AccountManagerImpl]
> (API-Job-Executor-16:ctx-e039eccb job-7312 ctx-40bb38cb) Access to
> VM[User|i-15-1151-VM] granted to
> Acct[a4e6eaed-9b06-4806-98f9-4fcc781b773f-graysonp] by DomainChecker
> 2015-04-17 13:46:22,614 DEBUG [c.c.n.NetworkModelImpl]
> (API-Job-Executor-16:ctx-e039eccb job-7312 ctx-40bb38cb) Service
> SecurityGroup is not supported in the network id=243
> 2015-04-17 13:46:22,621 DEBUG [c.c.n.NetworkModelImpl]
> (API-Job-Executor-16:ctx-e039eccb job-7312 ctx-40bb38cb) Service
> SecurityGroup is not supported in the network id=243
> 2015-04-17 13:46:22,658 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (API-Job-Executor-16:ctx-e039eccb job-7312 ctx-40bb38cb) Deploy avoids
> pods: [], clusters: [], hosts: []
> 2015-04-17 13:46:22,660 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (API-Job-Executor-16:ctx-e039eccb job-7312 ctx-40bb38cb)
> DeploymentPlanner allocation algorithm:
> com.cloud.deploy.FirstFitPlanner@6ba48686
> 2015-04-17 13:46:22,660 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (API-Job-Executor-16:ctx-e039eccb job-7312 ctx-40bb38cb) Trying to
> allocate a host and storage pools from dc:1, pod:null,cluster:null,
> requested cpu: 500, requested ram: 536870912
> 2015-04-17 13:46:22,660 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (API-Job-Executor-16:ctx-e039eccb job-7312 ctx-40bb38cb) Is ROOT
> volume READY (pool already allocated)?: No
> 2015-04-17 13:46:22,660 DEBUG [c.c.d.FirstFitPlanner]
> (API-Job-Executor-16:ctx-e039eccb job-7312 ctx-40bb38cb) Searching all
> possible resources under this Zone: 1
> 2015-04-17 13:46:22,662 DEBUG [c.c.d.FirstFitPlanner]
> (API-Job-Executor-16:ctx-e039eccb job-7312 ctx-40bb38cb) Listing
> clusters in order of aggregate capacity, that have (atleast one host
> with) enough CPU and RAM capacity under this Zone: 1
> 2015-04-17 13:46:22,668 DEBUG [c.c.d.FirstFitPlanner]
> (API-Job-Executor-16:ctx-e039eccb job-7312 ctx-40bb38cb) Removing from
> the clusterId list these clusters from avoid set: []
> 2015-04-17 13:46:22,678 DEBUG [c.c.d.FirstFitPlanner]
> (API-Job-Executor-16:ctx-e039eccb job-7312 ctx-40bb38cb) Cannot
> allocate cluster list [2] for vm creation since their allocated
> percentage crosses the disable capacity threshold defined at each
> cluster/ at global value for capacity Type : 1, skipping these
> clusters
> 2015-04-17 13:46:22,678 DEBUG [c.c.d.FirstFitPlanner]
> (API-Job-Executor-16:ctx-e039eccb job-7312 ctx-40bb38cb) No clusters
> found after removing disabled clusters and clusters in avoid list,
> returning.
> 2015-04-17 13:46:22,683 DEBUG [c.c.v.UserVmManagerImpl]
> (API-Job-Executor-16:ctx-e039eccb job-7312 ctx-40bb38cb) Destroying vm
> VM[User|i-15-1151-VM] as it failed to create on Host with Id:null
> 2015-04-17 13:46:22,706 DEBUG [c.c.c.CapacityManagerImpl]
> (API-Job-Executor-16:ctx-e039eccb job-7312 ctx-40bb38cb) VM state
> transitted from :Stopped to Error with event:
> OperationFailedToErrorvm's original host id: null new host id: null
> host id before state transition: null
> 2015-04-17 13:46:22,724 WARN  [o.a.c.alerts]
> (API-Job-Executor-16:ctx-e039eccb job-7312 ctx-40bb38cb)  alertType::
> 8 // dataCenterId:: 1 // podId:: null // clusterId:: null // message::
> Failed to deploy Vm with Id: 1151, on Host with Id: null
> 2015-04-17 13:46:22,780 INFO  [o.a.c.a.c.u.v.DeployVMCmd]
> (API-Job-Executor-16:ctx-e039eccb job-7312 ctx-40bb38cb)
> com.cloud.exception.InsufficientServerCapacityException: Unable to
> create a deployment for VM[User|i-15-1151-VM]Scope=interface
> com.cloud.dc.DataCenter; id=1
> 2015-04-17 13:46:22,781 INFO  [o.a.c.a.c.u.v.DeployVMCmd]
> (API-Job-Executor-16:ctx-e039eccb job-7312 ctx-40bb38cb) Unable to
> create a deployment for VM[User|i-15-1151-VM]
> com.cloud.exception.InsufficientServerCapacityException: Unable to
> create a deployment for VM[User|i-15-1151-VM]Scope=interface
> com.cloud.dc.DataCenter; id=1
> at
> org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.reserveVirtualMachine(VMEntityManagerImpl.java:214)
> at
> org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.reserve(VirtualMachineEntityImpl.java:200)
> at
> com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3468)
> at
> com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3123)
> at
> com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3111)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
> at
> org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:106)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
> at
> com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
> at
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
> at
> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
> at com.sun.proxy.$Proxy207.startVirtualMachine(Unknown Source)
> at
> org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:449)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:141)
> at
> com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:108)
> at
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:503)
> at
> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
> at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at
> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
> at
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:460)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 2015-04-17 13:46:22,783 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-16:ctx-e039eccb job-7312) Complete async job-7312,
> jobStatus: FAILED, resultCode: 530, result:
>
> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":533,"errortext":"Unable
> to create a deployment for VM[User|i-15-1151-VM]"}
> 2015-04-17 13:46:22,794 DEBUG [o.a.c.f.j.i.AsyncJobManage
>



-- 

Andrija Panić

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message