cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rajani Karuturi (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CLOUDSTACK-7986) [F5 LB] Failed to execute IPAssocCommand due to com.cloud.utils.exception.ExecutionException: Exception caught in Networking::urn:iControl:Networking/VLAN::create()
Date Mon, 08 Dec 2014 11:16:13 GMT

     [ https://issues.apache.org/jira/browse/CLOUDSTACK-7986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Rajani Karuturi updated CLOUDSTACK-7986:
----------------------------------------
    Fix Version/s: 4.3.2
                   4.4.3
                   4.6.0
                   4.5.0

> [F5 LB] Failed to execute IPAssocCommand due to com.cloud.utils.exception.ExecutionException:
Exception caught in Networking::urn:iControl:Networking/VLAN::create()
> --------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-7986
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-7986
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: Network Devices
>    Affects Versions: 4.3.0, 4.4.0, 4.5.0
>         Environment: Red Hat Enterprise Linux Server release 6.3 (Santiago)
> IG-IP 11.3.0 Build 39.0 VE Trial 11.3.0-HF1 (based on BIGIP 11.3.0HF6) 
>            Reporter: Sudhansu Sahu
>            Assignee: Sudhansu Sahu
>             Fix For: 4.5.0, 4.6.0, 4.4.3, 4.3.2
>
>
> When trying to launch a VM on a network that uses the network offering (with LB provided
by the F5) the following errors are observed in the logs and the creation ultimately fails:
> {noformat}
> 71831 2014-11-26 12:19:34,403 DEBUG [c.c.n.ExternalLoadBalancerDeviceManagerImpl] (Work-Job-Executor-133:ctx-434a0b57
job-269/job-270 ctx-cd8df97f) Allocated external load balancer device:1 for the network: 205
>  71832 2014-11-26 12:19:34,428 DEBUG [c.c.a.t.Request] (Work-Job-Executor-133:ctx-434a0b57
job-269/job-270 ctx-cd8df97f) Seq 4-5343239482898383377: Sending  { Cmd , MgmtId: 7407677735140,
via: 4(200-F5BigIpLoadBalancer-10.147.60.5       1), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.routing.IpAssocCommand":{"ipAddresses":[{"accountId":2,"sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":false,"broadcastUri":"910","vlanGateway":"10.0.160.37","vla
      nNetmask":"255.255.240.0","networkRate":200,"newNic":false}],"accessDetails":{},"wait":0}}]
}
>  71833 2014-11-26 12:19:34,429 DEBUG [c.c.a.t.Request] (Work-Job-Executor-133:ctx-434a0b57
job-269/job-270 ctx-cd8df97f) Seq 4-5343239482898383377: Executing:  { Cmd , MgmtId: 7407677735140,
via: 4(200-F5BigIpLoadBalancer-10.147.6       0.51), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.routing.IpAssocCommand":{"ipAddresses":[{"accountId":2,"sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":false,"broadcastUri":"910","vlanGateway":"10.0.160.37","
      vlanNetmask":"255.255.240.0","networkRate":200,"newNic":false}],"accessDetails":{},"wait":0}}]
}
>  71834 2014-11-26 12:19:34,429 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-217:ctx-40025c4d)
Seq 4-5343239482898383377: Executing request
>  71835 2014-11-26 12:19:34,731 DEBUG [c.c.n.r.F5BigIpResource] (DirectAgent-217:ctx-40025c4d)
Creating a guest VLAN with tag 910
>  71836 2014-11-26 12:19:35,616 DEBUG [c.c.a.ApiServlet] (catalina-exec-19:ctx-6bfff176)
===START===  10.252.192.47 -- GET  command=queryAsyncJobResult&jobId=33e182fe-ab28-470c-a483-a16bd40e05d9&response=json&sessionkey=cUbJKOPdiHo
      sLJ06cqS8y66FEMU%3D&_=1416984574574
>  71837 2014-11-26 12:19:35,711 ERROR [c.c.n.r.F5BigIpResource] (DirectAgent-217:ctx-40025c4d)
Failed to execute IPAssocCommand due to com.cloud.utils.exception.ExecutionException: Failed
to create vlan with tag 910
>  71838 2014-11-26 12:19:35,734 DEBUG [c.c.a.ApiServlet] (catalina-exec-19:ctx-6bfff176
ctx-f76b168b) ===END===  10.252.192.47 -- GET  command=queryAsyncJobResult&jobId=33e182fe-ab28-470c-a483-a16bd40e05d9&response=json&sessionkey=
      cUbJKOPdiHosLJ06cqS8y66FEMU%3D&_=1416984574574
>  71839 2014-11-26 12:19:35,842 ERROR [c.c.n.r.F5BigIpResource] (DirectAgent-217:ctx-40025c4d)
Retrying IpAssocCommand. Number of retries remaining: 1
>  71840 2014-11-26 12:19:36,053 DEBUG [c.c.n.r.F5BigIpResource] (DirectAgent-217:ctx-40025c4d)
Creating a guest VLAN with tag 910
>  71841 2014-11-26 12:19:36,233 ERROR [c.c.n.r.F5BigIpResource] (DirectAgent-217:ctx-40025c4d)
Exception caught in Networking::urn:iControl:Networking/VLAN::create()
>  71842 Exception: Common::OperationFailed
>  71843         primary_error_code   : 16908390 (0x01020066)
>  71844         secondary_error_code : 0
>  71845         error_string         : 01020066:3: The requested VLAN (/Common/vlan-910)
already exists in partition Common.
>  71846 2014-11-26 12:19:36,234 ERROR [c.c.n.r.F5BigIpResource] (DirectAgent-217:ctx-40025c4d)
Failed to execute IPAssocCommand due to com.cloud.utils.exception.ExecutionException: Exception
caught in Networking::urn:iControl:Netwo       rking/VLAN::create()
>  71847 Exception: Common::OperationFailed
>  71848         primary_error_code   : 16908390 (0x01020066)
>  71849         secondary_error_code : 0
>  71850         error_string         : 01020066:3: The requested VLAN (/Common/vlan-910)
already exists in partition Common.
>  71851 2014-11-26 12:19:36,318 ERROR [c.c.n.r.F5BigIpResource] (DirectAgent-217:ctx-40025c4d)
Retrying IpAssocCommand. Number of retries remaining: 0
>  71852 2014-11-26 12:19:36,529 DEBUG [c.c.n.r.F5BigIpResource] (DirectAgent-217:ctx-40025c4d)
Creating a guest VLAN with tag 910
>  71853 2014-11-26 12:19:36,562 ERROR [c.c.n.r.F5BigIpResource] (DirectAgent-217:ctx-40025c4d)
Exception caught in Networking::urn:iControl:Networking/VLAN::create()
>  71854 Exception: Common::OperationFailed
>  71855         primary_error_code   : 16908390 (0x01020066)
>  71856         secondary_error_code : 0
>  71857         error_string         : 01020066:3: The requested VLAN (/Common/vlan-910)
already exists in partition Common.
>  71858 2014-11-26 12:19:36,563 ERROR [c.c.n.r.F5BigIpResource] (DirectAgent-217:ctx-40025c4d)
Failed to execute IPAssocCommand due to com.cloud.utils.exception.ExecutionException: Exception
caught in Networking::urn:iControl:Netwo       rking/VLAN::create()
>  71859 Exception: Common::OperationFailed
>  71860         primary_error_code   : 16908390 (0x01020066)
>  71861         secondary_error_code : 0
>  71862         error_string         : 01020066:3: The requested VLAN (/Common/vlan-910)
already exists in partition Common.
>  71863 2014-11-26 12:19:36,563 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-217:ctx-40025c4d)
Seq 4-5343239482898383377: Response Received:
>  71864 2014-11-26 12:19:36,563 DEBUG [c.c.a.t.Request] (DirectAgent-217:ctx-40025c4d)
Seq 4-5343239482898383377: Processing:  { Ans: , MgmtId: 7407677735140, via: 4, Ver: v1, Flags:
10, [{"com.cloud.agent.api.routing.IpAssocAnswer       ":{"results":["Failed"],"result":false,"wait":0}}]
}
>  71865 2014-11-26 12:19:36,563 DEBUG [c.c.a.t.Request] (Work-Job-Executor-133:ctx-434a0b57
job-269/job-270 ctx-cd8df97f) Seq 4-5343239482898383377: Received:  { Ans: , MgmtId: 7407677735140,
via: 4, Ver: v1, Flags: 10, { IpAssocAn       swer } }
>  71866 2014-11-26 12:19:36,563 ERROR [c.c.n.ExternalLoadBalancerDeviceManagerImpl] (Work-Job-Executor-133:ctx-434a0b57
job-269/job-270 ctx-cd8df97f) External load balancer was unable to implement the guest network
on the external        load balancer in zone bpkvmwstorezone
>  71867 2014-11-26 12:19:36,570 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Work-Job-Executor-133:ctx-434a0b57
job-269/job-270 ctx-cd8df97f) Cleaning up because we're unable to implement the network Ntwk[205|Guest|15]
>  71868 2014-11-26 12:19:36,589 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Work-Job-Executor-133:ctx-434a0b57
job-269/job-270 ctx-cd8df97f) Lock is acquired for network Ntwk[205|Guest|15] as a part of
network shutdown
>  71869 2014-11-26 12:19:36,597 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Work-Job-Executor-133:ctx-434a0b57
job-269/job-270 ctx-cd8df97f) Releasing 0 port forwarding rules for network id=205 as a part
of shutdownNetworkRules
>  71870 2014-11-26 12:19:36,598 DEBUG [c.c.n.f.FirewallManagerImpl] (Work-Job-Executor-133:ctx-434a0b57
job-269/job-270 ctx-cd8df97f) There are no rules to forward to the network elements
>  71871 2014-11-26 12:19:36,599 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Work-Job-Executor-133:ctx-434a0b57
job-269/job-270 ctx-cd8df97f) Releasing 0 static nat rules for network id=205 as a part of
shutdownNetworkRules
>  71872 2014-11-26 12:19:36,599 DEBUG [c.c.n.f.FirewallManagerImpl] (Work-Job-Executor-133:ctx-434a0b57
job-269/job-270 ctx-cd8df97f) There are no rules to forward to the network elements
>  71873 2014-11-26 12:19:36,601 DEBUG [c.c.n.l.LoadBalancingRulesManagerImpl] (Work-Job-Executor-133:ctx-434a0b57
job-269/job-270 ctx-cd8df97f) Revoking 0 Public load balancing rules for network id=205
>  71874 2014-11-26 12:19:36,601 DEBUG [c.c.n.l.LoadBalancingRulesManagerImpl] (Work-Job-Executor-133:ctx-434a0b57
job-269/job-270 ctx-cd8df97f) There are no Load Balancing Rules to forward to the network
elements
>  71875 2014-11-26 12:19:36,602 DEBUG [c.c.n.l.LoadBalancingRulesManagerImpl] (Work-Job-Executor-133:ctx-434a0b57
job-269/job-270 ctx-cd8df97f) Revoking 0 Internal load balancing rules for network id=205
>  71876 2014-11-26 12:19:36,603 DEBUG [c.c.n.l.LoadBalancingRulesManagerImpl] (Work-Job-Executor-133:ctx-434a0b57
job-269/job-270 ctx-cd8df97f) There are no Load Balancing Rules to forward to the network
elements
> 71877 2014-11-26 12:19:36,604 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Work-Job-Executor-133:ctx-434a0b57
job-269/job-270 ctx-cd8df97f) Releasing 0 firewall ingress rules for network id=205 as a part
of shutdownNetworkRules
>  71878 2014-11-26 12:19:36,604 DEBUG [c.c.n.f.FirewallManagerImpl] (Work-Job-Executor-133:ctx-434a0b57
job-269/job-270 ctx-cd8df97f) There are no rules to forward to the network elements
>  71879 2014-11-26 12:19:36,606 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Work-Job-Executor-133:ctx-434a0b57
job-269/job-270 ctx-cd8df97f) Releasing 0 firewall egress rules for network id=205 as a part
of shutdownNetworkRules
>  71880 2014-11-26 12:19:36,607 DEBUG [c.c.n.f.FirewallManagerImpl] (Work-Job-Executor-133:ctx-434a0b57
job-269/job-270 ctx-cd8df97f) There are no rules to forward to the network elements
>  71881 2014-11-26 12:19:36,609 DEBUG [c.c.n.r.RulesManagerImpl] (Work-Job-Executor-133:ctx-434a0b57
job-269/job-270 ctx-cd8df97f) Found 0 static nat rules to apply for network id 205
>  71882 2014-11-26 12:19:36,631 DEBUG [c.c.n.e.VirtualRouterElement] (Work-Job-Executor-133:ctx-434a0b57
job-269/job-270 ctx-cd8df97f) Virtual router elemnt doesn't need to associate ip addresses
on the backend; virtual router does       n't exist in the network 205
>  71883 2014-11-26 12:19:36,632 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Work-Job-Executor-133:ctx-434a0b57
job-269/job-270 ctx-cd8df97f) Sending network shutdown to F5BigIP
>  71884 2014-11-26 12:19:36,648 WARN  [c.c.n.ExternalLoadBalancerDeviceManagerImpl] (Work-Job-Executor-133:ctx-434a0b57
job-269/job-270 ctx-cd8df97f) Network shutdwon requested on external load balancer element,
which did not imple       ment the network. Either network implement failed half way through
or already network shutdown is completed. So just returning.
>  71885 2014-11-26 12:19:36,656 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Work-Job-Executor-133:ctx-434a0b57
job-269/job-270 ctx-cd8df97f) Sending network shutdown to VirtualRouter
>  71886 2014-11-26 12:19:36,673 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Work-Job-Executor-133:ctx-434a0b57
job-269/job-270 ctx-cd8df97f) Network id=205 is shutdown successfully, cleaning up corresponding
resources now.
>  71887 2014-11-26 12:19:36,676 DEBUG [c.c.n.g.GuestNetworkGuru] (Work-Job-Executor-133:ctx-434a0b57
job-269/job-270 ctx-cd8df97f) Releasing vnet for the network id=205
>  71888 2014-11-26 12:19:36,710 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Work-Job-Executor-133:ctx-434a0b57
job-269/job-270 ctx-cd8df97f) Lock is released for network Ntwk[205|Guest|15] as a part of
network shutdown
>  71889 2014-11-26 12:19:36,710 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Work-Job-Executor-133:ctx-434a0b57
job-269/job-270 ctx-cd8df97f) Lock is released for network id 205 as a part of network implement
>  71890 2014-11-26 12:19:36,711 INFO  [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-133:ctx-434a0b57
job-269/job-270 ctx-cd8df97f) Unable to contact resource.
>  71891 com.cloud.exception.ResourceUnavailableException: Resource [Network:205] is unreachable:
External load balancer was unable to implement the guest network on the external load balancer
in zone bpkvmwstorezone
>  71892         at com.cloud.network.ExternalLoadBalancerDeviceManagerImpl.manageGuestNetworkWithExternalLoadBalancer(ExternalLoadBalancerDeviceManagerImpl.java:1058)
>  71893         at com.cloud.network.element.F5ExternalLoadBalancerElement.implement(F5ExternalLoadBalancerElement.java:158)
>  71894         at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.implementNetworkElementsAndResources(NetworkOrchestrator.java:1101)
>  71895         at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.implementNetwork(NetworkOrchestrator.java:1008)
>  71896         at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.implementNetwork(NetworkOrchestrator.java:938)
>  71897         at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare(NetworkOrchestrator.java:1293)
>  71898         at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:983)
>  71899         at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:4436)
>  71900         at sun.reflect.GeneratedMethodAccessor300.invoke(Unknown Source)
>  71901         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  71902         at java.lang.reflect.Method.invoke(Method.java:601)
>  71903         at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
>  71904         at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:4592)
>  71905         at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:103)
>  71906         at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:536)
>  71907         at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>  71908         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>  71909         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>  71910         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>  71911         at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>  71912         at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:493)
>  71913         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>  71914         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>  71915         at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>  71916         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>  71917         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>  71918         at java.lang.Thread.run(Thread.java:722)
>  71919 2014-11-26 12:19:36,712 WARN  [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-133:ctx-434a0b57
job-269/job-270 ctx-cd8df97f) unexpected ResourceUnavailableException : com.cloud.network.Network
>  71920 com.cloud.exception.ResourceUnavailableException: Resource [Network:205] is unreachable:
External load balancer was unable to implement the guest network on the external load balancer
in zone bpkvmwstorezone
>  71921         at com.cloud.network.ExternalLoadBalancerDeviceManagerImpl.manageGuestNetworkWithExternalLoadBalancer(ExternalLoadBalancerDeviceManagerImpl.java:1058)
>  71922         at com.cloud.network.element.F5ExternalLoadBalancerElement.implement(F5ExternalLoadBalancerElement.java:158)
>  71923         at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.implementNetworkElementsAndResources(NetworkOrchestrator.java:1101)
>  71924         at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.implementNetwork(NetworkOrchestrator.java:1008)
>  71925         at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.implementNetwork(NetworkOrchestrator.java:938)
>  71926         at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare(NetworkOrchestrator.java:1293)
>  71927         at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:983)
>  71928         at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:4436)
>  71929         at sun.reflect.GeneratedMethodAccessor300.invoke(Unknown Source)
>  71930         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  71931         at java.lang.reflect.Method.invoke(Method.java:601)
>  71932         at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
>  71933         at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:4592)
>  71934         at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:103)
>  71935         at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:536)
> 71936         at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>  71937         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>  71938         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>  71939         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>  71940         at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>  71941         at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:493)
>  71942         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>  71943         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>  71944         at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>  71945         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>  71946         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>  71947         at java.lang.Thread.run(Thread.java:722)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message