cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ilya musayev <ilya.mailing.li...@gmail.com>
Subject Re: Unable to start a VM due to insufficient capacity
Date Wed, 17 Sep 2014 03:19:20 GMT
I've seen this before and had similar issues.

Instead of re-installing or dropping DB, you can delete all job_ids that 
are stuck.

For example, this is what i would see in my logs.

(1013,VmWorkJobQueue, 2375) is reaching concurrency limit 1
(1013,VmWorkJobQueue, 2375) is reaching concurrency limit 1
(1433,VmWorkJobQueue, 2470) is reaching concurrency limit 1
(1434,VmWorkJobQueue, 2471) is reaching concurrency limit 1
(1475,VmWorkJobQueue, 2483) is reaching concurrency limit 1
(1477,VmWorkJobQueue, 2484) is reaching concurrency limit 1
(1478,VmWorkJobQueue, 596) is reaching concurrency limit 1
(981,VmWorkJobQueue, 2371) is reaching concurrency limit 1

DELETE ALL FROM BELOW:
SELECT * FROM cloud.sync_queue_item where queue_id=981 or queue_id=1478 
or queue_id=1477 or queue_id=1475 or queue_id=1434 or queue_id=1433 or 
queue_id=1013 ;
SELECT * FROM cloud.sync_queue where id=1433 or id=1434 or id=1475 or 
id=1477 or id=1478 or id=981;

While i did SELECT statement, you can alter to DELETE.

Regards
ilya

On 9/16/14, 6:31 PM, Matthew Midgett wrote:
> Ugo, I am having an issue just like you. I can deploy the default centos
> template but if I try to deploy a iso it looks like its failing in the
> network area. I've been reading your emails to Geoff and I have restarted
> the network with a clean and that didn't fix my problem. I also restarted
> the management servers during this process. I don't know what happed but
> ssvm and console quit working. I've spent all day deleting the whole thing
> and setting it back up again. Just to find that it's not working.
>
> Now I just saw this in the logs
>
> Cannot delete or update a parent row: a foreign key constraint fails
> (`cloud`.`async_job_join_map`, CONSTRAINT
> `fk_async_job_join_map__join_job_id` FOREIGN KEY (`join_job_id`) REFERENCES
> `async_job` (`id`))
>
> I am running a 2 node haproxy with virtual ip in front of a galera sql
> cluster.
>
> I dropped the DB, gave the cloud user more permissions. Deleted everything
> and restarted and it's doing this all over again. I am running xenserver 6.2
> and acs4.4. I even tried 4.3 again and it started doing the same thing.
>
>
>
> -----Original Message-----
> From: Ugo Vasi [mailto:ugo.vasi@procne.it]
> Sent: Tuesday, September 16, 2014 5:36 AM
> To: users@cloudstack.apache.org
> Subject: Re: Unable to start a VM due to insufficient capacity
>
> Hi Geoff,
> I restarted the network and now I can create new VM without errors.
>
> Now, how can I prevent this issue?
>
>
>
>
> Il 11/09/2014 09:28, Geoff Higginbottom ha scritto:
>> Hi Ugo,
>>
>> Did restarting the network with cleanup option resolve your issue last
> time, if yes have you tried the same on this network?
>> Regards
>>
>> Geoff Higginbottom
>> CTO / Cloud Architect
>>
>> D: +44 20 3603 0542<tel:+442036030542> | S: +44 20 3603
>> 0540<tel:+442036030540> | M: +447968161581<tel:+447968161581>
>>
>> geoff.higginbottom@shapeblue.com<mailto:geoff.higginbottom@shapeblue.c
>> om> | www.shapeblue.com<htp://www.shapeblue.com/> |
>> Twitter:@cloudstackguru<https://twitter.com/#!/cloudstackguru>
>>
>> ShapeBlue Ltd, 53 Chandos Place, Covent Garden, London, WC2N
>> 4HS<x-apple-data-detectors://5>
>>
>>
>> On 11 Sep 2014, at 08:17, "Ugo Vasi"
> <ugo.vasi@procne.it<mailto:ugo.vasi@procne.it>> wrote:
>> Hi Geoff,
>> I still have the problem creating new VMs but in another isolated network.
> The problem seems to be the same:
>> 2014-09-11 08:55:27,048 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) VM is being created in
>> podId: 1
>> 2014-09-11 08:55:27,050 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Network id=212 is already
>> implemented
>> 2014-09-11 08:55:27,118 DEBUG [c.c.n.NetworkModelImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Service SecurityGroup is
>> not supported in the network id=212
>> 2014-09-11 08:55:27,122 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Changing active number of
>> nics for network id=212 on 1
>> 2014-09-11 08:55:27,174 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Asking VirtualRouter to
>> prepare for Nic[116-86-8429f13f-7c3b-4b8a-aea3-9b8e9a3f4c03-10.1.1.19]
>> 2014-09-11 08:55:27,181 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Lock is acquired for
>> network id 212 as a part of router startup in
>> Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Po
>> ol(Id))] :
>> Dest[Zone(1)-Pod(1)-Cluster(1)-Host(7)-Storage(Volume(98|ROOT-->Pool(6
>> ))]
>> 2014-09-11 08:55:27,183 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Lock is released for
>> network id 212 as a part of router startup in
>> Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Po
>> ol(Id))] :
>> Dest[Zone(1)-Pod(1)-Cluster(1)-Host(7)-Storage(Volume(98|ROOT-->Pool(6
>> ))]
>> 2014-09-11 08:55:27,189 DEBUG [c.c.n.NetworkModelImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Service SecurityGroup is
>> not supported in the network id=212
>> 2014-09-11 08:55:27,198 DEBUG [c.c.n.NetworkModelImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Service SecurityGroup is
>> not supported in the network id=212
>> 2014-09-11 08:55:27,202 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Applying dhcp entry in
>> network Ntwk[212|Guest|8]
>> 2014-09-11 08:55:27,215 DEBUG [c.c.a.t.Request]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Seq 7-564304056: Sending
>> { Cmd , MgmtId: 77916390353436, via: 7(csh1b), Ver: v1, Flags: 100011,
>> [{"com.cloud.agent.api.routing.DhcpEntryCommand":{"vmMac":"02:00:15:76
>> :00:0e","vmIpAddress":"10.1.1.19","vmName":"giannitest","defaultRouter
>> ":"10.1.1.1","defaultDns":"10.1.1.1","duid":"00:03:00:01:02:00:15:76:0
>> 0:0e","isDefault":true,"executeInSequence":false,"accessDetails":{"rou
>> ter.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.name":
>> "r-60-VM","router.ip":"169.254.1.230"},"wait":0}}] }
>> 2014-09-11 08:55:27,941 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
>> (secstorage-1:ctx-327037b2) Zone 1 is ready to launch secondary
>> storage VM
>> 2014-09-11 08:55:28,022 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>> (consoleproxy-1:ctx-a8ba3b1e) Zone 1 is ready to launch console proxy
>> 2014-09-11 08:55:28,820 DEBUG [c.c.a.ApiServlet]
>> (catalina-exec-16:ctx-58e92382) ===START===  192.168.130.89 -- GET
>> command=queryAsyncJobResult&jobId=195bf80a-9823-4ec3-add4-8907b98fbaf0
>> &response=json&sessionkey=qGZA6KYoc1SB%2Bq2k70HoM%2BBBQTU%3D&_=1410418
>> 528731
>> 2014-09-11 08:55:29,069 DEBUG [c.c.a.t.Request]
>> (AgentManager-Handler-13:null) Seq 7-564304056: Processing:  { Ans: ,
>> MgmtId: 77916390353436, via: 7, Ver: v1, Flags: 10,
>> [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
>> 2014-09-11 08:55:29,069 DEBUG [c.c.a.t.Request]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Seq 7-564304056: Received:
>> { Ans: , MgmtId: 77916390353436, via: 7, Ver: v1, Flags: 10, { Answer
>> } }
>> 2014-09-11 08:55:29,075 DEBUG [c.c.n.NetworkModelImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Service SecurityGroup is
>> not supported in the network id=212
>> 2014-09-11 08:55:29,078 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Applying userdata and
>> password entry in network Ntwk[212|Guest|8]
>> 2014-09-11 08:55:29,089 DEBUG [c.c.a.t.Request]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Seq 7-564304057: Sending
>> { Cmd , MgmtId: 77916390353436, via: 7(csh1b), Ver: v1, Flags: 100011,
>> [{"com.cloud.agent.api.routing.SavePasswordCommand":{,"vmIpAddress":"1
>> 0.1.1.19","vmName":"giannitest","executeInSequence":false,"accessDetai
>> ls":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","rout
>> er.ip":"169.254.1.230","router.name":"r-60-VM"},"wait":0}},{"com.cloud
>> .agent.api.routing.VmDataCommand":{"vmIpAddress":"10.1.1.19","vmName":
>> "giannitest","executeInSequence":false,"accessDetails":{"router.guest.
>> ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.1.2
>> 30","router.name":"r-60-VM"},"wait":0}}] }
>> 2014-09-11 08:55:29,399 DEBUG [c.c.a.t.Request]
>> (AgentManager-Handler-10:null) Seq 7-564304057: Processing:  { Ans: ,
>> MgmtId: 77916390353436, via: 7, Ver: v1, Flags: 10,
>> [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}},{"com.cloud.a
>> gent.api.Answer":{"result":false,"details":"VmDataCommand failed,
>> check agent logs","wait":0}}] }
>> 2014-09-11 08:55:29,491 DEBUG [c.c.a.t.Request]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Seq 7-564304057: Received:
>> { Ans: , MgmtId: 77916390353436, via: 7, Ver: v1, Flags: 10, { Answer,
>> Answer } }
>> 2014-09-11 08:55:29,491 INFO  [c.c.v.VirtualMachineManagerImpl]
> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Unable to contact resource.
>> com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1]
> is unreachable: Unable to apply userdata and password entry on router
>>          at
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyRules(Virtu
> alNetworkApplianceManagerImpl.java:3915)
>>          at
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyUserData(Vi
> rtualNetworkApplianceManagerImpl.java:3118)
>>          at sun.reflect.GeneratedMethodAccessor472.invoke(Unknown
>> Source) ...
>> 2014-09-11 08:55:29,755 DEBUG [c.c.a.ApiServlet]
>> (catalina-exec-16:ctx-58e92382 ctx-ad8ddf67) ===END=== 192.168.130.89
>> -- GET
>> command=queryAsyncJobResult&jobId=195bf80a-9823-4ec3-add4-8907b98fbaf0
>> &response=json&sessionkey=qGZA6KYoc1SB%2Bq2k70HoM%2BBBQTU%3D&_=1410418
>> 528731
>> 2014-09-11 08:55:29,798 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Cleaning up resources for
>> the vm VM[User|giannitest] in Starting state
>> 2014-09-11 08:55:29,800 DEBUG [c.c.a.t.Request]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Seq 7-564304058: Sending
>> { Cmd , MgmtId: 77916390353436, via: 7(csh1b), Ver: v1, Flags: 100011,
>> [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequenc
>> e":false,"vmName":"i-2-86-VM","wait":0}}] }
>> 2014-09-11 08:55:29,950 DEBUG [c.c.a.t.Request]
>> (AgentManager-Handler-6:null) Seq 7-564304058: Processing:  { Ans: ,
>> MgmtId: 77916390353436, via: 7, Ver: v1, Flags: 10,
>> [{"com.cloud.agent.api.StopAnswer":{"result":true,"wait":0}}] }
>> 2014-09-11 08:55:29,950 DEBUG [c.c.a.t.Request]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Seq 7-564304058: Received:
>> { Ans: , MgmtId: 77916390353436, via: 7, Ver: v1, Flags: 10, {
>> StopAnswer } }
>> 2014-09-11 08:55:29,955 DEBUG [c.c.n.NetworkModelImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Service SecurityGroup is
>> not supported in the network id=212
>> 2014-09-11 08:55:29,957 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Changing active number of
>> nics for network id=212 on -1
>> 2014-09-11 08:55:30,051 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Asking VirtualRouter to
>> release
>> NicProfile[116-86-8429f13f-7c3b-4b8a-aea3-9b8e9a3f4c03-10.1.1.19-null
>> 2014-09-11 08:55:30,051 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Successfully released
>> network resources for the vm VM[User|giannitest]
>> 2014-09-11 08:55:30,051 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Successfully cleanued up
>> resources for the vm VM[User|giannitest] in Starting state
>> 2014-09-11 08:55:30,058 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Deploy avoids pods: [],
>> clusters: [], hosts: [7]
>> 2014-09-11 08:55:30,058 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) DataCenter id = '1' provided is
> in avoid set, DeploymentPlanner cannot allocate the VM, returning.
>> 2014-09-11 08:55:30,214 DEBUG [c.c.c.CapacityManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) VM state transitted from
>> :Starting to Stopped with event: OperationFailedvm's original host id:
>> null new host id: null host id before state transition: 7
>> 2014-09-11 08:55:30,218 DEBUG [c.c.c.CapacityManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Hosts's actual total CPU:
>> 36256 and CPU after applying overprovisioning: 36256
>> 2014-09-11 08:55:30,218 DEBUG [c.c.c.CapacityManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Hosts's actual total RAM:
>> 50639785984 and RAM after applying overprovisioning: 50639785984
>> 2014-09-11 08:55:30,218 DEBUG [c.c.c.CapacityManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) release cpu from host: 7,
>> old used: 26500,reserved: 0, actual total: 36256, total with
>> overprovisioning: 36256; new used: 26000,reserved:0;
>> movedfromreserved: false,moveToReserveredfalse
>> 2014-09-11 08:55:30,218 DEBUG [c.c.c.CapacityManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) release mem from host: 7,
>> old used: 21072183296,reserved: 0, total: 50639785984; new used:
>> 20535312384,reserved:0; movedfromreserved: false,moveToReserveredfalse
>> 2014-09-11 08:55:30,302 DEBUG [c.c.c.CapacityManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) VM state transitted from
>> :Stopped to Starting with event: StartRequestedvm's original host id:
>> null new host id: null host id before state transition: null
>> 2014-09-11 08:55:30,302 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Successfully transitioned
>> to start state for VM[User|giannitest] reservation id =
>> 4b6c2f8d-a34c-4b86-a0d4-acd5ea87353a
>> 2014-09-11 08:55:30,340 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Trying to deploy VM, vm
>> has dcId: 1 and podId: 1
>> 2014-09-11 08:55:30,340 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Deploy avoids pods: [],
>> clusters: [], hosts: [7]
>> 2014-09-11 08:55:30,348 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Deploy avoids pods: [],
>> clusters: [], hosts: [7]
>> 2014-09-11 08:55:30,348 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) DataCenter id = '1' provided is
> in avoid set, DeploymentPlanner cannot allocate the VM, returning.
>> 2014-09-11 08:55:30,435 DEBUG [c.c.c.CapacityManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) VM state transitted from
>> :Starting to Stopped with event: OperationFailedvm's original host id:
>> null new host id: null host id before state transition: null
>> 2014-09-11 08:55:30,477 DEBUG [c.c.v.UserVmManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Destroying vm
>> VM[User|giannitest] as it failed to create on Host with Id:null
>> 2014-09-11 08:55:30,609 DEBUG [c.c.c.CapacityManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) VM state transitted from
>> :Stopped to Error with event: OperationFailedToErrorvm's original host
>> id: null new host id: null host id before state transition: null
>> 2014-09-11 08:55:29,755 DEBUG [c.c.a.ApiServlet]
>> (catalina-exec-16:ctx-58e92382 ctx-ad8ddf67) ===END=== 192.168.130.89
>> -- GET
>> command=queryAsyncJobResult&jobId=195bf80a-9823-4ec3-add4-8907b98fbaf0
>> &response=json&sessionkey=qGZA6KYoc1SB%2Bq2k70HoM%2BBBQTU%3D&_=1410418
>> 528731
>> 2014-09-11 08:55:29,798 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Cleaning up resources for
>> the vm VM[User|giannitest] in Starting state
>> 2014-09-11 08:55:29,800 DEBUG [c.c.a.t.Request]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Seq 7-564304058: Sending
>> { Cmd , MgmtId: 77916390353436, via: 7(csh1b), Ver: v1, Flags: 100011,
>> [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequenc
>> e":false,"vmName":"i-2-86-VM","wait":0}}] }
>> 2014-09-11 08:55:29,950 DEBUG [c.c.a.t.Request]
>> (AgentManager-Handler-6:null) Seq 7-564304058: Processing:  { Ans: ,
>> MgmtId: 77916390353436, via: 7, Ver: v1, Flags: 10,
>> [{"com.cloud.agent.api.StopAnswer":{"result":true,"wait":0}}] }
>> 2014-09-11 08:55:29,950 DEBUG [c.c.a.t.Request]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Seq 7-564304058: Received:
>> { Ans: , MgmtId: 77916390353436, via: 7, Ver: v1, Flags: 10, {
>> StopAnswer } }
>> 2014-09-11 08:55:29,955 DEBUG [c.c.n.NetworkModelImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Service SecurityGroup is
>> not supported in the network id=212
>> 2014-09-11 08:55:29,957 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Changing active number of
>> nics for network id=212 on -1
>> 2014-09-11 08:55:30,051 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Asking VirtualRouter to
>> release
>> NicProfile[116-86-8429f13f-7c3b-4b8a-aea3-9b8e9a3f4c03-10.1.1.19-null
>> 2014-09-11 08:55:30,051 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Successfully released
>> network resources for the vm VM[User|giannitest]
>> 2014-09-11 08:55:30,051 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Successfully cleanued up
>> resources for the vm VM[User|giannitest] in Starting state
>> 2014-09-11 08:55:30,058 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Deploy avoids pods: [],
>> clusters: [], hosts: [7]
>> 2014-09-11 08:55:30,058 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) DataCenter id = '1' provided is
> in avoid set, DeploymentPlanner cannot allocate the VM, returning.
>> 2014-09-11 08:55:30,214 DEBUG [c.c.c.CapacityManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) VM state transitted from
>> :Starting to Stopped with event: OperationFailedvm's original host id:
>> null new host id: null host id before state transition: 7
>> 2014-09-11 08:55:30,218 DEBUG [c.c.c.CapacityManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Hosts's actual total CPU:
>> 36256 and CPU after applying overprovisioning: 36256
>> 2014-09-11 08:55:30,218 DEBUG [c.c.c.CapacityManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Hosts's actual total RAM:
>> 50639785984 and RAM after applying overprovisioning: 50639785984
>> 2014-09-11 08:55:30,218 DEBUG [c.c.c.CapacityManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) release cpu from host: 7,
>> old used: 26500,reserved: 0, actual total: 36256, total with
>> overprovisioning: 36256; new used: 26000,reserved:0;
>> movedfromreserved: false,moveToReserveredfalse
>> 2014-09-11 08:55:30,218 DEBUG [c.c.c.CapacityManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) release mem from host: 7,
>> old used: 21072183296,reserved: 0, total: 50639785984; new used:
>> 20535312384,reserved:0; movedfromreserved: false,moveToReserveredfalse
>> 2014-09-11 08:55:30,302 DEBUG [c.c.c.CapacityManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) VM state transitted from
>> :Stopped to Starting with event: StartRequestedvm's original host id:
>> null new host id: null host id before state transition: null
>> 2014-09-11 08:55:30,302 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Successfully transitioned
>> to start state for VM[User|giannitest] reservation id =
>> 4b6c2f8d-a34c-4b86-a0d4-acd5ea87353a
>> 2014-09-11 08:55:30,340 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Trying to deploy VM, vm
>> has dcId: 1 and podId: 1
>> 2014-09-11 08:55:30,340 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Deploy avoids pods: [],
>> clusters: [], hosts: [7]
>> 2014-09-11 08:55:30,348 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Deploy avoids pods: [],
>> clusters: [], hosts: [7]
>> 2014-09-11 08:55:30,348 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) DataCenter id = '1' provided is
> in avoid set, DeploymentPlanner cannot allocate the VM, returning.
>> 2014-09-11 08:55:30,435 DEBUG [c.c.c.CapacityManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) VM state transitted from
>> :Starting to Stopped with event: OperationFailedvm's original host id:
>> null new host id: null host id before state transition: null
>> 2014-09-11 08:55:30,477 DEBUG [c.c.v.UserVmManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Destroying vm
>> VM[User|giannitest] as it failed to create on Host with Id:null
>> 2014-09-11 08:55:30,609 DEBUG [c.c.c.CapacityManagerImpl]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) VM state transitted from
>> :Stopped to Error with event: OperationFailedToErrorvm's original host
>> id: null new host id: null host id before state transition: null
>> 2014-09-11 08:55:30,809 WARN  [o.a.c.alerts]
>> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de)  alertType:: 8 //
>> dataCenterId:: 1 // podId:: 1 // clusterId:: null // message:: Failed
>> to deploy Vm with Id: 86, on Host with Id: null
>> 2014-09-11 08:55:30,932 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-1:null) Ping from 6
>> 2014-09-11 08:55:31,078 ERROR [c.c.a.ApiAsyncJobDispatcher]
>> (Job-Executor-55:ctx-b5c34ec7) Unexpected exception while executing
>> org.apache.cloudstack.api.command.user.vm.DeployVMCmd
>> com.cloud.utils.exception.CloudRuntimeException: Unable to start a VM due
> to insufficient capacity
>>          at
> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:
> 605)
>>          at
> org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirt
> ualMachine(VMEntityManagerImpl.java:237)
>>          at
> org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deplo
> y(VirtualMachineEntityImpl.java:207)
>>          at
> com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:35
> 81)
>>          at
> com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:31
> 61)
>>          at
> com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:31
> 47)
>>          at sun.reflect.GeneratedMethodAccessor587.invoke(Unknown
>> Source) ...
>>
>>
>> Il 01/08/2014 11:44, Geoff Higginbottom ha scritto:
>> Ugo,
>>
>> Login to the VR and tail the messages log then attempt to start the VM,
> you should see various messages re DHCP updating etc in the VR.  If you
> don't see the VR update try restarting the network Iain the 'cleanup'
> option, this will create a new VR.
>> Regards
>>
>> Geoff Higginbottom
>> CTO / Cloud Architect
>>
>> D: +44 20 3603 0542<tel:+442036030542> | S: +44 20 3603
>> 0540<tel:+442036030540> | M: +447968161581<tel:+447968161581>
>>
>> geoff.higginbottom@shapeblue.com<mailto:geoff.higginbottom@shapeblue.c
>> om><mailto:geoff.higginbottom@shapeblue.com> |
>> www.shapeblue.com<http://www.shapeblue.com><htp://www.shapeblue.com/>
>> | Twitter:@cloudstackguru<https://twitter.com/#!/cloudstackguru>
>>
>> ShapeBlue Ltd, 53 Chandos Place, Covent Garden, London, WC2N
>> 4HS<x-apple-data-detectors://5>
>>
>>
>> On 1 Aug 2014, at 09:06, "Ugo Vasi"
> <ugo.vasi@procne.it<mailto:ugo.vasi@procne.it><mailto:ugo.vasi@procne.it>>
> wrote:
>> Hi Geoff,
>> I tried restarting r-29-VM but the problem is still blocking creation of
> new VM in that isolated network.
>> Creating a new VM in another isolated network, it works. Where can I look
> at to find the problem?
>>
>>
>> Il 31/07/2014 21:53, Geoff Higginbottom ha scritto:
>> Ugo,
>>
>> Looks like CS is failing to update the VR with details for the new VM, try
> restarting the VR.
>> Regards
>>
>> Geoff Higginbottom
>> CTO / Cloud Architect
>>
>>
>> D: +44 20 3603 0542<tel:+442036030542> | S: +44 20 3603
>> 0540<tel:+442036030540> | M: +447968161581<tel:+447968161581>
>>
>> geoff.higginbottom@shapeblue.com<mailto:geoff.higginbottom@shapeblue.c
>> om><mailto:geoff.higginbottom@shapeblue.com><mailto:geoff.higginbottom
>> @shapeblue.com> |
>> www.shapeblue.com<http://www.shapeblue.com><http://www.shapeblue.com><
>> htp://www.shapeblue.com/>
>>
>> ShapeBlue Ltd, 53 Chandos Place, Covent Garden, London, WC2N
>> 4HS<x-apple-data-detectors://37>
>>
>>
>>
>> On 31 Jul 2014, at 15:40, "Ugo Vasi"
> <ugo.vasi@procne.it<mailto:ugo.vasi@procne.it><mailto:ugo.vasi@procne.it><ma
> ilto:ugo.vasi@procne.it>> wrote:
>> Hi everybody,
>> I installed a CloudStack 4.3.0 system with a manager (in a virtual
> machine), two NFS primary storage, one NFS secondary storare and two host
> agent on ubuntu distribution.
>> The cluster have one advanced zone without security group configuration
> with KVM hipervisors.
>> I create some VM but now does not allow me to create more giving the error
> message "Unable to start a VM due to insufficient capacity".
>> I am sure that the resources (ip, ram, cpu and storage) are enough to
> create new instances. In dashboard I see all ok.
>> >From the management-server.log I see this errors messages (see attachment
> for full version) when I try to deploy a new virtual machine with 1x1GHz
> cpu, 1GB or RAM and 20GB disk space:
>> 2014-07-31 14:59:11,983 DEBUG [c.c.a.t.Request]
>> (AgentManager-Handler-12:null) Seq 1-684594119: Processing:  { Ans: ,
>> MgmtId: 77916390353436, via: 1, Ver: v1, Flags: 10,
>> [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}},{"com.cloud.a
>> gent.api.Answer":{"result":false,"details":"VmDataCommand failed,
>> check agent logs","wait":0}}] }
>> 2014-07-31 14:59:11,983 INFO  [c.c.v.VirtualMachineManagerImpl]
> (Job-Executor-26:ctx-ab3aee38 ctx-d8600c16) Unable to contact resource.
>> com.cloud.exception.ResourceUnavailableException: Resource
>> [DataCenter:1] is unreachable: Unable to apply userdata and password
>> entry on router
>> 2014-07-31 14:59:12,453 DEBUG [c.c.c.CapacityManagerImpl]
>> (Job-Executor-26:ctx-ab3aee38 ctx-d8600c16) VM state transitted from
>> :Starting to Stopped with event: OperationFailedvm's original host id:
>> null new host id: null host id before state transition: 7
>> 2014-07-31 14:59:12,677 DEBUG [c.c.c.CapacityManagerImpl]
>> (Job-Executor-26:ctx-ab3aee38 ctx-d8600c16) VM state transitted from
>> :Starting to Stopped with event: OperationFailedvm's original host id:
>> null new host id: null host id before state transition: null
>> 2014-07-31 14:59:12,708 DEBUG [c.c.v.UserVmManagerImpl]
>> (Job-Executor-26:ctx-ab3aee38 ctx-d8600c16) Destroying vm
>> VM[User|csweb2] as it failed to create on Host with Id:null
>> 2014-07-31 14:59:12,738 DEBUG [c.c.c.CapacityManagerImpl]
>> (Job-Executor-26:ctx-ab3aee38 ctx-d8600c16) VM state transitted from
>> :Stopped to Error with event: OperationFailedToErrorvm's original host
>> id: null new host id: null host id before state transition: null
>> 2014-07-31 14:59:12,929 WARN  [o.a.c.alerts]
>> (Job-Executor-26:ctx-ab3aee38 ctx-d8600c16)  alertType:: 8 //
>> dataCenterId:: 1 // podId:: 1 // clusterId:: null // message:: Failed
>> to deploy Vm with Id: 70, on Host with Id: null
>> 2014-07-31 14:59:13,159 ERROR [c.c.a.ApiAsyncJobDispatcher]
>> (Job-Executor-26:ctx-ab3aee38) Unexpected exception while executing
>> org.apache.cloudstack.api.command.user.vm.DeployVMCmd
>> com.cloud.utils.exception.CloudRuntimeException: Unable to start a VM
>> due to insufficient capacity Caused by:
>> com.cloud.exception.InsufficientServerCapacityException: Unable to
>> create a deployment for VM[User|csweb2]Scope=interface
>> com.cloud.dc.DataCenter; id=1
>> 2014-07-31 14:59:13,161 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Job-Executor-26:ctx-ab3aee38) Complete async job-783, jobStatus:
>> FAILED, resultCode: 530, result:
>> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":
>> [],"errorcode":530,"errortext":"Unable to start a VM due to
>> insufficient capacity"}
>> 2014-07-31 14:59:46,280 DEBUG [c.c.v.VirtualMachineManagerImpl]
>> (Job-Executor-27:ctx-82be396e ctx-41f0cf24) Stopped called on
>> VM[User|csweb2] but the state is Error
>> 2014-07-31 14:59:46,330 DEBUG [c.c.c.CapacityManagerImpl]
>> (Job-Executor-27:ctx-82be396e ctx-41f0cf24) VM state transitted from
>> :Error to Expunging with event: DestroyRequestedvm's original host id:
>> null new host id: null host id before state transition: null
>>
>> I tried to search in others log file ("VmDataCommand failed, check agent
> logs"), but I don't found nothing...
>> Can anyone Help me?
>>
>> --
>>
>>    U g o   V a s i
> <ugo.vasi@procne.it<mailto:ugo.vasi@procne.it><mailto:ugo.vasi@procne.it><ma
> ilto:ugo.vasi@procne.it>>
>>    P r o c n e  s.r.l    >)
>>    via Cotonificio 45  33010 Tavagnacco IT
>>    phone: +390432486523 fax: +390432486523
>>
>> Le informazioni contenute in questo messaggio sono riservate e
>> confidenziali ed ? vietata la diffusione in qualunque modo eseguita.
>> Qualora Lei non fosse la persona a cui il presente messaggio ?
>> destinato, La invitiamo ad eliminarlo e a non leggerlo, dandocene
>> gentilmente comunicazione.
>> Per qualsiasi informazione si prega di contattare
> support@procne.it<mailto:support@procne.it><mailto:support@procne.it><mailto
> :support@procne.it> .
>> Rif. D.L. 196/2003
>>
>>
>>
>>
>> <management-server.log>
>> Find out more about ShapeBlue and our range of CloudStack related
>> services
>>
>> IaaS Cloud Design &
>> Build<http://shapeblue.com/iaas-cloud-design-and-build//>
>> CSForge - rapid IaaS deployment
>> framework<http://shapeblue.com/csforge/>
>> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
>> CloudStack Infrastructure
>> Support<http://shapeblue.com/cloudstack-infrastructure-support/>
>> CloudStack Bootcamp Training
>> Courses<http://shapeblue.com/cloudstack-training/>
>>
>> This email and any attachments to it may be confidential and are intended
> solely for the use of the individual to whom it is addressed. Any views or
> opinions expressed are solely those of the author and do not necessarily
> represent those of Shape Blue Ltd or related companies. If you are not the
> intended recipient of this email, you must neither take any action based
> upon its contents, nor copy or show it to anyone. Please contact the sender
> if you believe you have received this email in error. Shape Blue Ltd is a
> company incorporated in England & Wales. ShapeBlue Services India LLP is a
> company incorporated in India and is operated under license from Shape Blue
> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil
> and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a
> company registered by The Republic of South Africa and is traded under
> license from Shape Blue Ltd. ShapeBlue is a registered trademark.
>>
>>
>>
>>
>>
>> --
>>
>>    U g o   V a s i
> <ugo.vasi@procne.it<mailto:ugo.vasi@procne.it><mailto:ugo.vasi@procne.it>>
>>    P r o c n e  s.r.l    >)
>>    via Cotonificio 45  33010 Tavagnacco IT
>>    phone: +390432486523 fax: +390432486523
>>
>> Le informazioni contenute in questo messaggio sono riservate e
>> confidenziali ed ? vietata la diffusione in qualunque modo eseguita.
>> Qualora Lei non fosse la persona a cui il presente messaggio ?
>> destinato, La invitiamo ad eliminarlo e a non leggerlo, dandocene
>> gentilmente comunicazione.
>> Per qualsiasi informazione si prega di contattare
> support@procne.it<mailto:support@procne.it><mailto:support@procne.it> .
>> Rif. D.L. 196/2003
>>
>>
>>
>>
>>
>> Find out more about ShapeBlue and our range of CloudStack related
>> services
>>
>> IaaS Cloud Design &
>> Build<http://shapeblue.com/iaas-cloud-design-and-build//>
>> CSForge - rapid IaaS deployment
>> framework<http://shapeblue.com/csforge/>
>> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
>> CloudStack Infrastructure
>> Support<http://shapeblue.com/cloudstack-infrastructure-support/>
>> CloudStack Bootcamp Training
>> Courses<http://shapeblue.com/cloudstack-training/>
>>
>> This email and any attachments to it may be confidential and are intended
> solely for the use of the individual to whom it is addressed. Any views or
> opinions expressed are solely those of the author and do not necessarily
> represent those of Shape Blue Ltd or related companies. If you are not the
> intended recipient of this email, you must neither take any action based
> upon its contents, nor copy or show it to anyone. Please contact the sender
> if you believe you have received this email in error. Shape Blue Ltd is a
> company incorporated in England & Wales. ShapeBlue Services India LLP is a
> company incorporated in India and is operated under license from Shape Blue
> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil
> and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a
> company registered by The Republic of South Africa and is traded under
> license from Shape Blue Ltd. ShapeBlue is a registered trademark.
>>
>>
>>
>>
>>
>> --
>>
>>    U g o   V a s i    <ugo.vasi@procne.it<mailto:ugo.vasi@procne.it>>
>>    P r o c n e  s.r.l    >)
>>    via Cotonificio 45  33010 Tavagnacco IT
>>    phone: +390432486523 fax: +390432486523
>>
>> Le informazioni contenute in questo messaggio sono riservate e
>> confidenziali ed ? vietata la diffusione in qualunque modo eseguita.
>> Qualora Lei non fosse la persona a cui il presente messaggio ?
>> destinato, La invitiamo ad eliminarlo e a non leggerlo, dandocene
>> gentilmente comunicazione.
>> Per qualsiasi informazione si prega di contattare
> support@procne.it<mailto:support@procne.it> .
>> Rif. D.L. 196/2003
>>
>>
>> Find out more about ShapeBlue and our range of CloudStack related
>> services
>>
>> IaaS Cloud Design &
>> Build<http://shapeblue.com/iaas-cloud-design-and-build//>
>> CSForge - rapid IaaS deployment
>> framework<http://shapeblue.com/csforge/>
>> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
>> CloudStack Infrastructure
>> Support<http://shapeblue.com/cloudstack-infrastructure-support/>
>> CloudStack Bootcamp Training
>> Courses<http://shapeblue.com/cloudstack-training/>
>>
>> This email and any attachments to it may be confidential and are intended
> solely for the use of the individual to whom it is addressed. Any views or
> opinions expressed are solely those of the author and do not necessarily
> represent those of Shape Blue Ltd or related companies. If you are not the
> intended recipient of this email, you must neither take any action based
> upon its contents, nor copy or show it to anyone. Please contact the sender
> if you believe you have received this email in error. Shape Blue Ltd is a
> company incorporated in England & Wales. ShapeBlue Services India LLP is a
> company incorporated in India and is operated under license from Shape Blue
> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil
> and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a
> company registered by The Republic of South Africa and is traded under
> license from Shape Blue Ltd. ShapeBlue is a registered trademark.
>>
>>
>>
>


Mime
View raw message