Hi,
I am running ACS 4.8.0. One cluster with two xenserver 6.5 hosts.
One of the VM's network went unreachable so when I checked VR it was unable to reach gateway.
So I tried to restart VR and did cleanup. But issue is still same. When it rebooted VR, I
saw ping command to gateway which ending with this ,
PING 123.23.24.25 (123.23.24.25): 48 data bytes
/etc/init.d/cloud-early-config: line 574: 2009 Terminated ping -n -c 3 $GW
But then it did give root login prompt on console. But problem is I am unable to login to
VR anymore since cloudstack just shutdown VR vm from xen and I can't even log into VR console
so I can't check more logs.
Cloudstack shut down VR automatically with this mail alert -
Command: com.cloud.agent.api.Command failed while starting virtual router
Details: Timed out in waiting SSH execution result com.cloud.agent.api.Answer
And on cloudstack management logs I get this error,
2016-08-24 14:32:41,548 ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-112:ctx-3f255d1a
job-17880) (logid:0e364177) Unexpected exception while executing org.apache.cloudstack.api.command.admin.router.RebootRouterCmd
com.cloud.exception.AgentUnavailableException: Resource [Host:1] is unreachable: Host 1: Unable
to start instance due to Unable to start VM:772fed1c-bb75-4778-9811-7a0a3633cb5b due to error
in finalizeStart, not retrying
at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1120)
at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:4580)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:4741)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:554)
at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:502)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.cloud.utils.exception.ExecutionException: Unable to start VM:772fed1c-bb75-4778-9811-7a0a3633cb5b
due to error in finalizeStart, not retrying
at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1084)
at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:4580)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
... 18 more
2016-08-24 14:32:41,550 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-112:ctx-3f255d1a
job-17880) (logid:0e364177) Complete async job-17880, jobStatus: FAILED, resultCode: 530,
result: org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":530,"errortext":"Resource
[Host:1] is unreachable: Host 1: Unable to start instance due to Unable to start VM:772fed1c-bb75-4778-9811-7a0a3633cb5b
due to error in finalizeStart, not retrying"}
2016-08-24 14:32:41,553 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-112:ctx-3f255d1a
job-17880) (logid:0e364177) Publish async job-17880 complete on message bus
Network I am trying to get up is production network, so I need it up asap. What could be the
problem here?
I tried to create new VR, and that seems to reach gateway. Its problem with only this VR.
What if I destroy router, from cloudstack panel? Would it recreate VR automatically? Or any
other suggestion on this issue?
/sonali
|