cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Milamber (JIRA)" <j...@apache.org>
Subject [jira] [Closed] (CLOUDSTACK-9255) Unable to start VM DomainRouter due to error in finalizeStart, not retrying
Date Sun, 20 Mar 2016 14:56:33 GMT

     [ https://issues.apache.org/jira/browse/CLOUDSTACK-9255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Milamber closed CLOUDSTACK-9255.
--------------------------------
       Resolution: Fixed
         Assignee: Wilder Rodrigues
    Fix Version/s: 4.9.0
                   4.7.2

> Unable to start VM DomainRouter due to error in finalizeStart, not retrying
> ---------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-9255
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9255
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: Virtual Router
>    Affects Versions: 4.7.0, 4.6.2, 4.8.0, 4.7.1
>         Environment: Ubuntu 14.04.3
> KVM
> NFS (primary/secondary)
>            Reporter: Milamber
>            Assignee: Wilder Rodrigues
>             Fix For: 4.7.2, 4.9.0
>
>         Attachments: anon-rvr-2nd-after-20.log
>
>
> I've spent 3 days with the same issue : unable to restart with clean up a network (virtual
router or redondant virtual router) if the network have at least 20 virtual machines.
> I've tested with CS 4.6.2, 4.7.0, 4.7.1RC1, 4.8.0RC1, same problem. I've used the system
vm from apt-get.eu and last builds from jenkins.
> My tests are made with hosts/mgr on Ubuntu 14.04.3 / KVM / NFS primary/secondary.
> My test case (with ansible modules) :
> 1/ create a new network (normal or RVR)
> 2/ create 20 vms (same params, just the name is changes)
> wait the end of creation
> 3/ restart the network with clean up option
> 4/ wait the restart, after some minutes, an error message arrived : "Failed to restart
network"
> The trace in management.log are:
> 2016-01-23 23:02:51,503 ERROR [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-51:ctx-9ed51622
job-268/job-271) (logid:b9a521fa) Unable to complete AsyncJobVO {id:271, userId: 2, accountId:
2, instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo: rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAACAAAAAAAAAAIAAAAAAAAAMnQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwc3IAEWphdmEudXRpbC5IYXNoTWFwBQfawcMWYNEDAAJGAApsb2FkRmFjdG9ySQAJdGhyZXNob2xkeHA_QAAAAAAADHcIAAAAEAAAAAF0AA5SZXN0YXJ0TmV0d29ya3QAP3JPMEFCWE55QUJGcVlYWmhMbXhoYm1jdVFtOXZiR1ZoYnMwZ2NvRFZuUHJ1QWdBQldnQUZkbUZzZFdWNGNBRXhw,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid:
146456419427, completeMsid: null, lastUpdated: null, lastPolled: null, created: Sat Jan 23
22:56:00 CET 2016}, job origin:268
> com.cloud.exception.AgentUnavailableException: Resource [Host:1] is unreachable: Host
1: Unable to start instance due to Unable to start VM[DomainRouter|r-50-VM] due to error in
finalizeStart, not retrying
>     at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1119)
>     at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:4578)
>     at sun.reflect.GeneratedMethodAccessor374.invoke(Unknown Source)
>     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke(Method.java:606)
>     at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
>     at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:4734)
>     at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
>     at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:554)
>     at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>     at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>     at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>     at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>     at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>     at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:502)
>     at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>     at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>     at java.lang.Thread.run(Thread.java:745)
> Caused by: com.cloud.utils.exception.ExecutionException: Unable to start VM[DomainRouter|r-50-VM]
due to error in finalizeStart, not retrying
>     at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1083)
>     at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:4578)
>     at sun.reflect.GeneratedMethodAccessor374.invoke(Unknown Source)
>     ... 17 more
> During the restart of the network I can connect on the VR with link local link over ssh,
the last lines shows:
> 2016-01-23 22:02:39,780  configure.py __init__:128 AclIP created for rule ==> {'last_port':
65535, u'protocol': u'tcp', u'revoked': False, u'already_added': True, u'source_cidr_list':
[u'0.0.0.0/0'], 'cidr': [u'0.0.0.0/0'], u'id': 52, u'src_ip': u'192.168.13.30', u'purpose':
u'Firewall', 'allowed': True, 'action': 'ACCEPT', u'src_port_range': [1, 65535], u'traffic_type':
u'Ingress', 'type': u'tcp', u'default_egress_policy': False, 'first_port': 1}
> 2016-01-23 22:02:39,780  configure.py add_rule:165 Current ACL IP direction is ==>
ingress
> 2016-01-23 22:02:39,780  merge.py load:60 Loading data bag type forwardingrules
> Broadcast message from root@r-50-VM (Sat Jan 23 22:02:45 2016):
> The system is going down for system halt NOW!
> Broadcast message from root@r-50-VM (Sat Jan 23 22:02:45 2016):
> Power button pressed
> The system is going down for system halt NOW!
> /opt/cloud/bin/vr_cfg.sh: line 60: 16845 Killed                  /opt/cloud/bin/update_config.py
vm_metadata.json
> Sat Jan 23 22:02:46 UTC 2016 : VR config: executing failed: /opt/cloud/bin/update_config.py
vm_metadata.json
> Connection to 169.254.2.186 closed by remote host.
> Connection to 169.254.2.186 closed.
> Perhaps that was a timeout issue? if I create one VM or 10 VMs, the network restart works.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message