cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sugandh S <s.suga...@rocketmail.com>
Subject Re: vm stuck in starting state, unable to delete it
Date Thu, 20 Mar 2014 07:16:00 GMT
Hi,

So, I have finally managed to access console proxy vm but I am still unable to create instances.

I have pasted the log file here: http://pastebin.com/UrjLuBiM

Here are the exceptions from the log that I got:


2014-03-20 12:26:10,766 DEBUG [agent.transport.Request] (AgentManager-Handler-15:null) Seq
1-1879572498: Processing:  { Ans: , 
MgmtId: 279278805451363, via: 1, Ver: v1, Flags: 110, 
[{"com.cloud.agent.api.Answer":{"result":false,"details":"ssh: connect 
to host 169.254.3.71 port 3922: No route to host","wait":0}}] }
2014-03-20 12:26:10,766 DEBUG 
[agent.manager.AgentAttache] (AgentManager-Handler-15:null) Seq 
1-1879572498: No more commands found
2014-03-20 12:26:10,766 DEBUG [agent.transport.Request] (Job-Executor-1:job-15 = [ 838a99c8-ebe5-4ce6-bcf9-5187cddbd945
]) Seq 
1-1879572498: Received:  { Ans: , MgmtId: 279278805451363, via: 1, Ver: 
v1, Flags: 110, { Answer } }
2014-03-20 12:26:10,766 INFO 
 [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-1:job-15 = [ 
838a99c8-ebe5-4ce6-bcf9-5187cddbd945 ]) Unable to contact resource.
com.cloud.exception.ResourceUnavailableException: Resource [Pod:1] is unreachable: Unable
to apply dhcp entry on router 
        at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyRules(VirtualNetworkApplianceManagerImpl.java:3826)
        at 
com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyDhcpEntry(VirtualNetworkApplianceManagerImpl.java:2943)
        at com.cloud.network.element.VirtualRouterElement.addDhcpEntry(VirtualRouterElement.java:902)
        at com.cloud.network.NetworkManagerImpl.prepareElement(NetworkManagerImpl.java:2079)
        at com.cloud.network.NetworkManagerImpl.prepareNic(NetworkManagerImpl.java:2200)
        at com.cloud.network.NetworkManagerImpl.prepare(NetworkManagerImpl.java:2136)
        at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:886)
        at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:577)
        at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:227)
        at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3440)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3000)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2986)
        at 
com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
        at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:420)
        at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
        at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:701)
2014-03-20 12:26:10,859 DEBUG 
[cloud.vm.VirtualMachineManagerImpl] (Job-Executor-1:job-15 = [ 
838a99c8-ebe5-4ce6-bcf9-5187cddbd945 ]) Cleaning up resources for the vm VM[User|test] in
Starting state
2014-03-20 12:26:10,861 DEBUG [agent.transport.Request] (Job-Executor-1:job-15 = [ 838a99c8-ebe5-4ce6-bcf9-5187cddbd945
]) Seq 
1-1879572499: Sending  { Cmd , MgmtId: 279278805451363, via: 1, Ver: v1, Flags: 100111, 
[{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":true,"vmName":"i-2-10-VM","wait":0}}]
}

and this: 


2014-03-20 12:26:12,826 INFO  [user.vm.DeployVMCmd] 
(Job-Executor-1:job-15 = [ 838a99c8-ebe5-4ce6-bcf9-5187cddbd945 ]) 
com.cloud.exception.InsufficientServerCapacityException: Unable to 
create a deployment for VM[User|test]Scope=interface 
com.cloud.dc.DataCenter; id=1
2014-03-20 12:26:12,826 INFO  [user.vm.DeployVMCmd] 
(Job-Executor-1:job-15 = [ 838a99c8-ebe5-4ce6-bcf9-5187cddbd945 ]) 
Unable to create a deployment for VM[User|test]
com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for
VM[User|test]Scope=interface 
com.cloud.dc.DataCenter; id=1
        at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:841)
        at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:577)
        at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:237)
        at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3440)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3000)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2986)
        at 
com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
        at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:420)
        at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
        at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:701)
2014-03-20 12:26:12,828 DEBUG 
[cloud.async.AsyncJobManagerImpl] (Job-Executor-1:job-15 = [ 
838a99c8-ebe5-4ce6-bcf9-5187cddbd945 ]) Complete async job-15 = [ 
838a99c8-ebe5-4ce6-bcf9-5187cddbd945 ], jobStatus: 2, resultCode: 530, 
result: Error Code: 533 Error text: Unable to create a deployment for 
VM[User|test]




On Thursday, 20 March 2014 9:16 AM, Sugandh S <s.sugandh@rocketmail.com> wrote:
 
Hi,

Yes, I did try that but it gave me this error "ssh: connect to host 169.254.2.158 port 3922:
No route to
 host".

Sugandh





On Wednesday, 19 March 2014 7:16 PM, Hugues Lepesant <hugues@lepesant.com> wrote:

Hi,

 
Do you try to ssh from the host running KVM and hosting the SystemVM ?

 
hug
 

-----Message initial-----
De:Sugandh S <s.sugandh@rocketmail.com>
Envoyé:mer. 19-03-2014 13:00
Sujet:Re: vm stuck in starting state, unable to delete it
À:Rajesh Battala <rajesh.battala@citrix.com>; users@cloudstack.apache.org; Sailaja
Mada <sailaja.mada@citrix.com>; Suresh Sadhu <Suresh.Sadhu@citrix.com>; 
Well, the router state seems to be "running" but I am not able to ping any of the system vms
via their link local ip address or their public or private ips.


When I try to ssh into console proxy vm, I get this:
root@server2:~# ssh -i /root/.ssh/id_rsa.cloud  -p 3922 root@169.254.2.158
ssh: connect to host 169.254.2.158 port 3922: No route to host


Thanks,
Sugandh




On Wednesday, 19 March 2014 5:31 PM, Rajesh Battala <rajesh.battala@citrix.com> wrote:

>From the log, issue is while applying the dhcp entry in the VR hence deployment fails.
Can you check whether the VR is up and network is in implemented state.?
 
Thanks
Rajesh Battala
 
From:Sugandh S [mailto:s.sugandh@rocketmail.com] 
Sent: Wednesday, March 19, 2014 5:07 PM
To: Rajesh Battala; users@cloudstack.apache.org; Sailaja Mada; Suresh Sadhu
Subject: Re: vm stuck in starting state, unable to delete it
 
Hi,

It took a couple of reboots to get system vms and router working again but now I have got
another problem, whenever I create an instance I get "Unable to create a deployment for VM[User|<vmname>]"
error.

I have pasted the log here:
http://tny.cz/1ee21d5e
 
 
On Wednesday, 19 March 2014 4:39 PM, Rajesh Battala <rajesh.battala@citrix.com> wrote:
Can you just capture the log from when you started the action till you see the error. 

Thanks
Rajesh Battala

-----Original Message-----
From: Sugandh S [mailto:s.sugandh@rocketmail.com] 
Sent: Wednesday, March 19, 2014 4:23 PM
To: Sailaja Mada; users@cloudstack.apache.org; Suresh Sadhu
Subject: Re: vm stuck in starting state, unable to
 delete it

Hi,

The log file is around 10gb, I used pastebinit to upload it and it gave me "memory error".

Is there any other way to provide the log file?

Thanks,
Sugandh




On Wednesday, 19 March 2014 3:56 PM, Sailaja Mada <sailaja.mada@citrix.com> wrote:

Hi,
 
Can you please send the complete log using PasteBin.     
 
Thanks,
Sailaja.M
 
From:Sugandh S [mailto:s.sugandh@rocketmail.com]
Sent: 19 March 2014 15:30
To: users@cloudstack.apache.org; Sailaja Mada; Suresh Sadhu; Sugandh S
Subject: Re: vm stuck in starting state, unable to delete it
 
Hi,
 
I just noticed that my domain router is also stuck in starting state and one of the vms I
created is now showing error state.
 
On Wednesday, 19 March 2014 3:25 PM, Sugandh S <s.sugandh@rocketmail.com> wrote:
Hi,

> I have noticed VM in starting state when Template is getting  Copied  from
 Secondary Storage to    > Primary Storage . 

It's been over 150 minutes and I don't think it should take this long to copy the template.

> size of the template and also value of global config parameter "wait" 

size of the iso is 700.29 MB and "wait" value is default "1800".

Sugandh




On Wednesday, 19 March 2014 3:16 PM, Sailaja Mada <sailaja.mada@citrix.com> wrote:

Hi,

I have noticed VM in starting state when Template is getting  Copied  from Secondary Storage
to Primary Storage . 

VM gets deployed and will move to
 running state after this copy is completed. Can you please share the size of the template
and also value of global config parameter "wait" 

One reason could be Storage Server is slow and Copy operation is taking longer time.  It
would help not to time out if you increase the "wait" value . But you may have to wait for
the copy operation to complete to get the VM into running state.

Thanks,
Sailaja.M


-----Original Message-----
From: Sugandh S [mailto:s.sugandh@rocketmail.com]
Sent: 19 March 2014 15:01
To: Suresh Sadhu; users@cloudstack.apache.org
Subject: Re: vm stuck in starting state, unable to delete it

Here is another part of log

e to Agent:3, com.cloud.exception.OperationTimedoutException: Commands 1710489660 to Host
3 timed out after 3600
2014-03-19 13:59:22,969 ERROR [cloud.server.StatsCollector] (StatsCollector-1:null) Error
trying to retrieve storage sta ts
com.cloud.utils.exception.CloudRuntimeException: Failed to send command, due to Agent:3, com.cloud.exception.OperationTi
medoutException: Commands 1710489660 to Host 3 timed out after 3600
2014-03-19 13:59:58,120 INFO  [utils.exception.CSExceptionErrorCode] (StatsCollector-1:null)
Could not find exception: c om.cloud.exception.OperationTimedoutException in error code list
for exceptions
2014-03-19 13:59:58,120 WARN  [agent.manager.AgentAttache] (StatsCollector-1:null)
 Seq 4-698744901: Timed out on null
2014-03-19 13:59:58,120 WARN  [agent.manager.AgentManagerImpl] (StatsCollector-1:null) Operation
timed out: Commands 698
744901 to Host 4 timed out after 3600
2014-03-19 13:59:58,122 WARN  [cloud.resource.ResourceManagerImpl] (StatsCollector-1:null)
Unable to obtain host 4 stati stics. 
2014-03-19 13:59:58,122 WARN  [cloud.server.StatsCollector] (StatsCollector-1:null) Received
invalid host stats for host
: 4
2014-03-19 14:00:07,507 WARN  [apache.cloudstack.alerts] (HA-2:null)  alertType:: 13 //
dataCenterId:: 0 // podId:: 0 //
 clusterId:: null // message:: No usage server process running
2014-03-19 14:00:22,973 INFO  [utils.exception.CSExceptionErrorCode] (StatsCollector-1:null)
Could not find exception: c om.cloud.exception.OperationTimedoutException in
 error code list for exceptions
2014-03-19 14:00:22,973 WARN  [agent.manager.AgentAttache] (StatsCollector-1:null) Seq 3-1710489661:
Timed out on null
2014-03-19 14:00:22,973 DEBUG [cloudstack.storage.RemoteHostEndPoint] (StatsCollector-1:null)
Failed to send command, du e to Agent:3, com.cloud.exception.OperationTimedoutException: Commands
1710489661 to Host 3 timed out after 3600
2014-03-19 14:00:22,973 ERROR [cloud.server.StatsCollector] (StatsCollector-1:null) Error
trying to retrieve storage sta ts
com.cloud.utils.exception.CloudRuntimeException: Failed to send command, due to Agent:3, com.cloud.exception.OperationTi
medoutException: Commands 1710489661 to Host 3 timed out after 3600
2014-03-19 14:00:58,715 INFO  [utils.exception.CSExceptionErrorCode] (StatsCollector-2:null)
Could not find exception: c
 om.cloud.exception.OperationTimedoutException in error code list for exceptions
2014-03-19 14:00:58,715 WARN  [agent.manager.AgentAttache] (StatsCollector-2:null) Seq 4-698744902:
Timed out on null
2014-03-19 14:00:58,716 WARN  [agent.manager.AgentManagerImpl] (StatsCollector-2:null) Operation
timed out: Commands 698
744902 to Host 4 timed out after 3600
2014-03-19 14:00:58,716 WARN  [cloud.resource.ResourceManagerImpl] (StatsCollector-2:null)
Unable to obtain host 4 stati stics. 
2014-03-19 14:00:58,716 WARN  [cloud.server.StatsCollector] (StatsCollector-2:null) Received
invalid host stats for host
: 4
2014-03-19 14:01:22,978 INFO  [utils.exception.CSExceptionErrorCode] (StatsCollector-2:null)
Could not find exception: c om.cloud.exception.OperationTimedoutException in error code list
for exceptions
2014-03-19 14:01:22,978 WARN  [agent.manager.AgentAttache] (StatsCollector-2:null) Seq 3-1710489662:
Timed out on null
2014-03-19 14:01:22,978 DEBUG [cloudstack.storage.RemoteHostEndPoint] (StatsCollector-2:null)
Failed to send command, du e to Agent:3, com.cloud.exception.OperationTimedoutException: Commands
1710489662 to Host 3 timed out after 3600
2014-03-19 14:01:22,978 ERROR [cloud.server.StatsCollector] (StatsCollector-2:null) Error
trying to retrieve storage sta ts
com.cloud.utils.exception.CloudRuntimeException: Failed to send command, due to Agent:3, com.cloud.exception.OperationTimedoutException:
Commands 1710489662 to Host 3 timed out after 3600
2014-03-19 14:01:35,900 WARN  [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-1:job-1
= [ d2584efe-01e9-42bf-a5a1-e9871265e5b3 ]) The task item for vm VM[DomainRouter|r-4-VM] has
been inactive for 3660
2014-03-19 14:01:35,900 ERROR [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-1:job-1 =
[ d2584efe-01e9-42bf-a5a1-e9871265e5b3 ]) Failed to start instance VM[User|Ubuntu-mysql]
com.cloud.utils.exception.CloudRuntimeException: Unable to start a VM due to concurrent operation
Caused by: com.cloud.exception.ConcurrentOperationException: There are concurrent operations
on VM[DomainRouter|r-4-VM]
2014-03-19 14:01:59,312 INFO  [utils.exception.CSExceptionErrorCode] (StatsCollector-1:null)
Could not find exception: com.cloud.exception.OperationTimedoutException in error code list
for exceptions
2014-03-19 14:01:59,312 WARN  [agent.manager.AgentAttache] (StatsCollector-1:null) Seq 4-698744904:
Timed out on null
2014-03-19 14:01:59,312 WARN  [agent.manager.AgentManagerImpl] (StatsCollector-1:null) Operation
timed out: Commands 698744904 to Host 4 timed out after
 3600
2014-03-19 14:01:59,312 WARN  [cloud.resource.ResourceManagerImpl] (StatsCollector-1:null)
Unable to obtain host 4 statistics. 
2014-03-19 14:01:59,312 WARN  [cloud.server.StatsCollector] (StatsCollector-1:null) Received
invalid host stats for host: 4
2014-03-19 14:02:07,765 WARN  [cloud.vm.VirtualMachineManagerImpl] (HA-Worker-0:work-5) The
task item for vm VM[DomainRouter|r-4-VM] has been inactive for 3720
2014-03-19 14:02:07,766 WARN  [cloud.ha.HighAvailabilityManagerImpl] (HA-Worker-0:work-5)
Failed to deploy vm 4 with original planner, sending HAPlanner
2014-03-19 14:02:07,768 DEBUG [cloud.vm.VirtualMachineManagerImpl] (HA-Worker-0:work-5) Unable
to transition into Starting state due to Unable to transition to a new state from Starting
via StartRequested
2014-03-19 14:02:07,811 DEBUG [cloud.vm.VirtualMachineManagerImpl]
 (HA-Worker-0:work-5) Determining why we're unable to update the state to Starting for VM[DomainRouter|r-4-VM]. 
Retry=4
2014-03-19 14:02:07,812 WARN  [cloud.vm.VirtualMachineManagerImpl] (HA-Worker-0:work-5) The
task item for vm VM[DomainRouter|r-4-VM] has been inactive for 3720
2014-03-19 14:02:07,812 WARN  [cloud.ha.HighAvailabilityManagerImpl] (HA-Worker-0:work-5)
Unable to restart VM[DomainRouter|r-4-VM] due to There are concurrent operations on VM[DomainRouter|r-4-VM]
2014-03-19 14:02:07,812 WARN  [apache.cloudstack.alerts] (HA-Worker-0:work-5)  alertType::
9 // dataCenterId:: 1 // podId:: 1 // clusterId:: null // message:: Unable to restart r-4-VM
which was running on host name: server2(id:1), availability zone: zone1,
pod: pod1
2014-03-19 14:02:22,983 INFO  [utils.exception.CSExceptionErrorCode] (StatsCollector-3:null)
Could not find
 exception: com.cloud.exception.OperationTimedoutException in error code list for exceptions
2014-03-19 14:02:22,983 WARN  [agent.manager.AgentAttache] (StatsCollector-3:null) Seq 3-1710489663:
Timed out on null
2014-03-19 14:02:22,983 DEBUG [cloudstack.storage.RemoteHostEndPoint] (StatsCollector-3:null)
Failed to send command, due to Agent:3, com.cloud.exception.OperationTimedoutException: Commands
1710489663 to Host 3 timed out after 3600
2014-03-19 14:02:22,983 ERROR [cloud.server.StatsCollector] (StatsCollector-3:null) Error
trying to retrieve storage stats
com.cloud.utils.exception.CloudRuntimeException: Failed to send command, due to Agent:3, com.cloud.exception.OperationTimedoutException:
Commands 1710489663 to Host 3 timed out after 3600
2014-03-19 14:02:59,904 INFO  [utils.exception.CSExceptionErrorCode] (StatsCollector-3:null)
Could not find exception:
 com.cloud.exception.OperationTimedoutException in error code list for exceptions
2014-03-19 14:02:59,904 WARN  [agent.manager.AgentAttache] (StatsCollector-3:null) Seq 4-698744905:
Timed out on null
2014-03-19 14:02:59,905 WARN  [agent.manager.AgentManagerImpl] (StatsCollector-3:null) Operation
timed out: Commands 698
744905 to Host 4 timed out after 3600
2014-03-19 14:02:59,905 WARN  [cloud.resource.ResourceManagerImpl] (StatsCollector-3:null)
Unable to obtain host 4 statistics. 
2014-03-19 14:02:59,905 WARN  [cloud.server.StatsCollector] (StatsCollector-3:null) Received
invalid host stats for host: 4
2014-03-19 14:03:22,988 INFO  [utils.exception.CSExceptionErrorCode] (StatsCollector-3:null)
Could not find exception: com.cloud.exception.OperationTimedoutException in error code list
for exceptions
2014-03-19
 14:03:22,988 WARN  [agent.manager.AgentAttache] (StatsCollector-3:null) Seq 3-1710489664:
Timed out on null
2014-03-19 14:03:22,988 DEBUG [cloudstack.storage.RemoteHostEndPoint] (StatsCollector-3:null)
Failed to send command, due to Agent:3, com.cloud.exception.OperationTimedoutException: Commands
1710489664 to Host 3 timed out after 3600
2014-03-19 14:03:22,988 ERROR [cloud.server.StatsCollector] (StatsCollector-3:null) Error
trying to retrieve storage stats
com.cloud.utils.exception.CloudRuntimeException: Failed to send command, due to Agent:3, com.cloud.exception.OperationTimedoutException:
Commands 1710489664 to Host 3 timed out after 3600
2014-03-19 14:04:00,500 INFO  [utils.exception.CSExceptionErrorCode] (StatsCollector-1:null)
Could not find exception: com.cloud.exception.OperationTimedoutException in error code list
for exceptions
2014-03-19 14:04:00,500
 WARN  [agent.manager.AgentAttache] (StatsCollector-1:null) Seq 4-698744906: Timed out on
null
2014-03-19 14:04:00,501 WARN  [agent.manager.AgentManagerImpl] (StatsCollector-1:null) Operation
timed out: Commands 698744906 to Host 4 timed out after 3600
2014-03-19 14:04:00,501 WARN  [cloud.resource.ResourceManagerImpl] (StatsCollector-1:null)
Unable to obtain host 4 statistics. 
2014-03-19 14:04:00,501 WARN  [cloud.server.StatsCollector] (StatsCollector-1:null) Received
invalid host stats for host: 4
2014-03-19 14:04:22,993 INFO  [utils.exception.CSExceptionErrorCode] (StatsCollector-3:null)
Could not find exception: com.cloud.exception.OperationTimedoutException in error code list
for exceptions
2014-03-19 14:04:22,993 WARN  [agent.manager.AgentAttache] (StatsCollector-3:null) Seq 3-1710489665:
Timed out on null
2014-03-19
 14:04:22,993 DEBUG [cloudstack.storage.RemoteHostEndPoint] (StatsCollector-3:null) Failed
to send command, due to Agent:3, com.cloud.exception.OperationTimedoutException: Commands
1710489665 to Host 3 timed out after 3600
2014-03-19 14:04:22,993 ERROR [cloud.server.StatsCollector] (StatsCollector-3:null) Error
trying to retrieve storage stats
com.cloud.utils.exception.CloudRuntimeException: Failed to send command, due to Agent:3, com.cloud.exception.OperationTimedoutException:
Commands 1710489665 to Host 3 timed out after 3600
2014-03-19 14:05:01,092 INFO  [utils.exception.CSExceptionErrorCode] (StatsCollector-2:null)
Could not find exception: com.cloud.exception.OperationTimedoutException in error code list
for exceptions
2014-03-19 14:05:01,092 WARN  [agent.manager.AgentAttache] (StatsCollector-2:null) Seq 4-698744907:
Timed out on null
2014-03-19 14:05:01,092
 WARN  [agent.manager.AgentManagerImpl] (StatsCollector-2:null) Operation timed out: Commands
698744907 to Host 4 timed out after 3600
2014-03-19 14:05:01,092 WARN  [cloud.resource.ResourceManagerImpl] (StatsCollector-2:null)
Unable to obtain host 4 statistics. 
2014-03-19 14:05:01,092 WARN  [cloud.server.StatsCollector] (StatsCollector-2:null) Received
invalid host stats for host
: 4
2014-03-19 14:05:23,000 INFO  [utils.exception.CSExceptionErrorCode] (StatsCollector-3:null)
Could not find exception: com.cloud.exception.OperationTimedoutException in error code list
for exceptions
2014-03-19 14:05:23,000 WARN  [agent.manager.AgentAttache] (StatsCollector-3:null) Seq 3-1710489666:
Timed out on null
2014-03-19 14:05:23,000 DEBUG [cloudstack.storage.RemoteHostEndPoint] (StatsCollector-3:null)
Failed to send command, due to Agent:3,
 com.cloud.exception.OperationTimedoutException: Commands 1710489666 to Host 3 timed out after
3600
2014-03-19 14:05:23,000 ERROR [cloud.server.StatsCollector] (StatsCollector-3:null) Error
trying to retrieve storage stats
com.cloud.utils.exception.CloudRuntimeException: Failed to send command, due to Agent:3, com.cloud.exception.OperationTimedoutException:
Commands 1710489666 to Host 3 timed out after 3600
2014-03-19 14:06:01,680 INFO  [utils.exception.CSExceptionErrorCode] (StatsCollector-1:null)
Could not find exception: com.cloud.exception.OperationTimedoutException in error code list
for exceptions
2014-03-19 14:06:01,680 WARN  [agent.manager.AgentAttache] (StatsCollector-1:null) Seq 4-698744908:
Timed out on null
2014-03-19 14:06:01,680 WARN  [agent.manager.AgentManagerImpl] (StatsCollector-1:null) Operation
timed out: Commands 698744908 to Host 4 timed
 out after 3600
2014-03-19 14:06:01,680 WARN  [cloud.resource.ResourceManagerImpl] (StatsCollector-1:null)
Unable to obtain host 4 statistics. 
2014-03-19 14:06:01,681 WARN  [cloud.server.StatsCollector] (StatsCollector-1:null) Received
invalid host stats for host: 4
2014-03-19 14:06:23,004 INFO  [utils.exception.CSExceptionErrorCode] (StatsCollector-1:null)
Could not find exception: com.cloud.exception.OperationTimedoutException in error code list
for exceptions
2014-03-19 14:06:23,004 WARN  [agent.manager.AgentAttache] (StatsCollector-1:null) Seq 3-1710489667:
Timed out on null
2014-03-19 14:06:23,004 DEB





On Wednesday, 19 March 2014 2:36 PM, Suresh Sadhu <Suresh.Sadhu@citrix.com> wrote:

Can you please  provide the logs and also  did  you notice  any exception in the management
log.


For deleting vm :
You can update the vm state in db  as Stopped and try to delete them from CS.

Regards
Sadhu




-----Original Message-----
From: Sugandh S [mailto:s.sugandh@rocketmail.com]
Sent: 19 March 2014 14:30
To: users@cloudstack.apache.org
Subject: vm stuck
 in starting state, unable to delete it

Hello all,

I am using CS 4.2 and my setup is as follows:

One server, running Ubuntu 12.04, is serving as both Cloudstack-management server and Cloudstack-agent.
Primary storage and secondary storage are also provided by this server via NFS. For primary
storage, export location is /export/primary and for secondary 
storage, it is /export/secondary.

Second server, also running Ubuntu 12.04, only serves as Cloudstack-agent. 


Now, when I create vms they are stuck in starting state and I am unable to delete them.

Any and all help would be greatly appreciated.

Thanks ahead,
Sugandh
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message