cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chandan Purushothama (JIRA)" <j...@apache.org>
Subject [jira] [Created] (CLOUDSTACK-7424) [Automation] Failed to restart network
Date Tue, 26 Aug 2014 00:04:57 GMT
Chandan Purushothama created CLOUDSTACK-7424:
------------------------------------------------

             Summary: [Automation] Failed to restart network
                 Key: CLOUDSTACK-7424
                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-7424
             Project: CloudStack
          Issue Type: Bug
      Security Level: Public (Anyone can view this level - this is the default.)
          Components: Automation, Test
    Affects Versions: 4.5.0
            Reporter: Chandan Purushothama
            Priority: Critical
             Fix For: 4.5.0



Error Executing the test case *integration.component.test_multiple_ips_per_nic.TestVmNetworkOperations.test_network_restart_cleanup_false_3_VPC*

*Error Message*

Job failed: {jobprocstatus : 0, created : u'2014-08-24T07:31:03+0000', cmd : u'org.apache.cloudstack.api.command.user.network.RestartNetworkCmd',
userid : u'77df13d2-2b4d-11e4-89bd-1e5d0e053e75', jobstatus : 2, jobid : u'01faacdc-294b-4209-a855-2dc8ea0aa2b4',
jobresultcode : 530, jobresulttype : u'object', jobresult : {errorcode : 530, errortext :
u'Failed to restart network'}, accountid : u'77df055e-2b4d-11e4-89bd-1e5d0e053e75'}   Logs
available at http://xenrt.hq.xensource.com/control/queue.cgi?action=testlogs&id=804076&phase=Parallel&test=multiple_ips_per_n

*Stacktrace*

  File "/usr/lib/python2.7/unittest/case.py", line 332, in run
    testMethod()
  File "/usr/local/lib/python2.7/dist-packages/ddt.py", line 114, in wrapper
    return func(self, *args, **kwargs)
  File "/root/cloudstack/test/integration/component/test_multiple_ips_per_nic.py", line 1186,
in test_network_restart_cleanup_false
    network.restart(self.apiclient, cleanup=False)
  File "/usr/local/lib/python2.7/dist-packages/marvin/lib/base.py", line 2626, in restart
    return(apiclient.restartNetwork(cmd))
  File "/usr/local/lib/python2.7/dist-packages/marvin/cloudstackAPI/cloudstackAPIClient.py",
line 1899, in restartNetwork
    response = self.connection.marvinRequest(command, response_type=response, method=method)
  File "/usr/local/lib/python2.7/dist-packages/marvin/cloudstackConnection.py", line 379,
in marvinRequest
    raise e
'Job failed: {jobprocstatus : 0, created : u\'2014-08-24T07:31:03+0000\', cmd : u\'org.apache.cloudstack.api.command.user.network.RestartNetworkCmd\',
userid : u\'77df13d2-2b4d-11e4-89bd-1e5d0e053e75\', jobstatus : 2, jobid : u\'01faacdc-294b-4209-a855-2dc8ea0aa2b4\',
jobresultcode : 530, jobresulttype : u\'object\', jobresult : {errorcode : 530, errortext
: u\'Failed to restart network\'}, accountid : u\'77df055e-2b4d-11e4-89bd-1e5d0e053e75\'}\n

Logs available at http://xenrt.hq.xensource.com/control/queue.cgi?action=testlogs&id=804076&phase=Parallel&test=multiple_ips_per_n
	

==============================
*Error in the Management Server Log*:
==============================

{code}
2014-08-24 07:31:03,602 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (API-Job-Executor-82:ctx-6738c61c
job-1529 ctx-3eaba8e6) Router r-229-VM is in Stopping, so not sending apply static nat rules
commands to the backend
2014-08-24 07:31:03,621 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (API-Job-Executor-82:ctx-6738c61c
job-1529 ctx-3eaba8e6) Router r-229-VM is in Stopping, so not sending apply vpc ip association
commands to the backend
2014-08-24 07:31:03,638 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (API-Job-Executor-82:ctx-6738c61c
job-1529 ctx-3eaba8e6) Router r-229-VM is in Stopping, so not sending apply vpc ip association
commands to the backend
2014-08-24 07:31:03,640 DEBUG [o.a.c.e.o.NetworkOrchestrator] (API-Job-Executor-82:ctx-6738c61c
job-1529 ctx-3eaba8e6) Sending network shutdown to VpcVirtualRouter
2014-08-24 07:31:03,653 DEBUG [c.c.n.NetworkModelImpl] (API-Job-Executor-82:ctx-6738c61c job-1529
ctx-3eaba8e6) Service SecurityGroup is not supported in the network id=314
2014-08-24 07:31:03,654 DEBUG [c.c.n.r.VpcVirtualNetworkApplianceManagerImpl] (API-Job-Executor-82:ctx-6738c61c
job-1529 ctx-3eaba8e6) Router r-229-VM is in Stopping, so not sending setup guest network
command to the backend
2014-08-24 07:31:03,663 DEBUG [c.c.v.VirtualMachineManagerImpl] (API-Job-Executor-82:ctx-6738c61c
job-1529 ctx-3eaba8e6) Lock is acquired for nic id 476 as a part of remove vm VM[DomainRouter|r-229-VM]
from network Ntwk[314|Guest|37]
2014-08-24 07:31:03,667 DEBUG [c.c.n.NetworkModelImpl] (API-Job-Executor-82:ctx-6738c61c job-1529
ctx-3eaba8e6) Service SecurityGroup is not supported in the network id=314
2014-08-24 07:31:03,668 WARN  [c.c.v.VirtualMachineManagerImpl] (API-Job-Executor-82:ctx-6738c61c
job-1529 ctx-3eaba8e6) Unable to remove vm VM[DomainRouter|r-229-VM] from network  Ntwk[314|Guest|37]
2014-08-24 07:31:03,669 DEBUG [c.c.v.VirtualMachineManagerImpl] (API-Job-Executor-82:ctx-6738c61c
job-1529 ctx-3eaba8e6) Lock is released for nic id 476 as a part of remove vm VM[DomainRouter|r-229-VM]
from network Ntwk[314|Guest|37]
2014-08-24 07:31:03,671 WARN  [o.a.c.e.o.NetworkOrchestrator] (API-Job-Executor-82:ctx-6738c61c
job-1529 ctx-3eaba8e6) Unable to complete shutdown of the network elements due to element:
VpcVirtualRouter
com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1] is unreachable:
Unable to remove vm VM[DomainRouter|r-229-VM] from network, is not in the right state
	at com.cloud.vm.VirtualMachineManagerImpl.orchestrateRemoveVmFromNetwork(VirtualMachineManagerImpl.java:3079)
	at com.cloud.vm.VirtualMachineManagerImpl.removeVmFromNetwork(VirtualMachineManagerImpl.java:3009)
	at com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.removeVpcRouterFromGuestNetwork(VpcVirtualNetworkApplianceManagerImpl.java:321)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
	at $Proxy190.removeVpcRouterFromGuestNetwork(Unknown Source)
	at com.cloud.network.element.VpcVirtualRouterElement.shutdown(VpcVirtualRouterElement.java:261)
	at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.shutdownNetworkElementsAndResources(NetworkOrchestrator.java:2161)
	at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.restartNetwork(NetworkOrchestrator.java:2468)
	at com.cloud.network.NetworkServiceImpl.restartNetwork(NetworkServiceImpl.java:1874)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
	at org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:106)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
	at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
	at $Proxy157.restartNetwork(Unknown Source)
	at org.apache.cloudstack.api.command.user.network.RestartNetworkCmd.execute(RestartNetworkCmd.java:95)
	at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:141)
	at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:108)
	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:503)
	at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
	at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:460)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
	at java.util.concurrent.FutureTask.run(FutureTask.java:166)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
	at java.lang.Thread.run(Thread.java:722)
2014-08-24 07:31:03,673 DEBUG [o.a.c.e.o.NetworkOrchestrator] (API-Job-Executor-82:ctx-6738c61c
job-1529 ctx-3eaba8e6) Failed to shutdown the network elements and resources as a part of
network restart: Implemented
2014-08-24 07:31:03,673 DEBUG [o.a.c.e.o.NetworkOrchestrator] (API-Job-Executor-82:ctx-6738c61c
job-1529 ctx-3eaba8e6) Marking network Ntwk[314|Guest|37] with restartRequired=true
2014-08-24 07:31:03,675 WARN  [c.c.n.NetworkServiceImpl] (API-Job-Executor-82:ctx-6738c61c
job-1529 ctx-3eaba8e6) Network id=314 failed to restart.
2014-08-24 07:31:03,680 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-82:ctx-6738c61c
job-1529) Complete async job-1529, jobStatus: FAILED, resultCode: 530, result: org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":530,"errortext":"Failed
to restart network"}
{code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message