That did not helped, I ended up creating new network, so we changed dns. That was very weird issue. /Sonali -----Original Message----- From: Glenn Wagner [mailto:glenn.wagner@shapeblue.com] Sent: Wednesday, August 24, 2016 8:59 PM To: users@cloudstack.apache.org Subject: RE: VR unable to reach gateway Ok if you destroy the VR , cloudstack will create a new one. Glenn glenn.wagner@shapeblue.com www.shapeblue.com First Floor, Victoria Centre, 7 Victoria Street, Somerset West, Cape Town 7129South Africa @shapeblue -----Original Message----- From: Sonali Jadhav [mailto:sonali@servercentralen.se] Sent: Wednesday, 24 August 2016 5:20 PM To: users@cloudstack.apache.org Subject: RE: VR unable to reach gateway Hi, Yes I tried to move VR to another Host, that also gave same issue. And no Hosts are unreachable, that was weird error. I have tried making new VR, for test that worked. /Sonali -----Original Message----- From: Glenn Wagner [mailto:glenn.wagner@shapeblue.com] Sent: Wednesday, August 24, 2016 8:42 PM To: users@cloudstack.apache.org Subject: RE: VR unable to reach gateway Hi. The other VR that works which host is it on? The error seems to be a HOST 1 problem - [Host:1] is unreachable: Can you migrate the VR to the other host and see if it works ? Glenn glenn.wagner@shapeblue.com www.shapeblue.com First Floor, Victoria Centre, 7 Victoria Street, Somerset West, Cape Town 7129South Africa @shapeblue -----Original Message----- From: Sonali Jadhav [mailto:sonali@servercentralen.se] Sent: Wednesday, 24 August 2016 4:53 PM To: users@cloudstack.apache.org Subject: VR unable to reach gateway Hi, I am running ACS 4.8.0. One cluster with two xenserver 6.5 hosts. One of the VM's network went unreachable so when I checked VR it was unable to reach gateway. So I tried to restart VR and did cleanup. But issue is still same. When it rebooted VR, I saw ping command to gateway which ending with this , PING 123.23.24.25 (123.23.24.25): 48 data bytes /etc/init.d/cloud-early-config: line 574: 2009 Terminated ping -n -c 3 $GW But then it did give root login prompt on console. But problem is I am unable to login to VR anymore since cloudstack just shutdown VR vm from xen and I can't even log into VR console so I can't check more logs. Cloudstack shut down VR automatically with this mail alert - Command: com.cloud.agent.api.Command failed while starting virtual router Details: Timed out in waiting SSH execution result com.cloud.agent.api.Answer And on cloudstack management logs I get this error, 2016-08-24 14:32:41,548 ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-112:ctx-3f255d1a job-17880) (logid:0e364177) Unexpected exception while executing org.apache.cloudstack.api.command.admin.router.RebootRouterCmd com.cloud.exception.AgentUnavailableException: Resource [Host:1] is unreachable: Host 1: Unable to start instance due to Unable to start VM:772fed1c-bb75-4778-9811-7a0a3633cb5b due to error in finalizeStart, not retrying at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1120) at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:4580) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107) at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:4741) at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:554) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:502) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: com.cloud.utils.exception.ExecutionException: Unable to start VM:772fed1c-bb75-4778-9811-7a0a3633cb5b due to error in finalizeStart, not retrying at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1084) at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:4580) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ... 18 more 2016-08-24 14:32:41,550 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-112:ctx-3f255d1a job-17880) (logid:0e364177) Complete async job-17880, jobStatus: FAILED, resultCode: 530, result: org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":530,"errortext":"Resource [Host:1] is unreachable: Host 1: Unable to start instance due to Unable to start VM:772fed1c-bb75-4778-9811-7a0a3633cb5b due to error in finalizeStart, not retrying"} 2016-08-24 14:32:41,553 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-112:ctx-3f255d1a job-17880) (logid:0e364177) Publish async job-17880 complete on message bus Network I am trying to get up is production network, so I need it up asap. What could be the problem here? I tried to create new VR, and that seems to reach gateway. Its problem with only this VR. What if I destroy router, from cloudstack panel? Would it recreate VR automatically? Or any other suggestion on this issue? /sonali