Mailing-List: contact cloudstack-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: cloudstack-dev@incubator.apache.org
Date: Wed, 19 Sep 2012 23:13:08 +1100 (NCT)
From: "Abhinav Roy (JIRA)" <jira@apache.org>
To: cloudstack-dev@incubator.apache.org
Message-ID: <1694882713.97054.1348056788645.JavaMail.jiratomcat@arcas>
In-Reply-To: <1349249901.89326.1347912247697.JavaMail.jiratomcat@arcas>
Subject: [jira] [Commented] (CLOUDSTACK-127) Adding VMs to an existing LB
 rule after upgrading from CS 3.0.2 to ASF 4.0 fails with
 "ResourceUnavailableException: Resource [DataCenter:1] is unreachable:
 Unable to apply ip association on router "
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/CLOUDSTACK-127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13458616#comment-13458616 ] 

Abhinav Roy commented on CLOUDSTACK-127:
----------------------------------------

Yes , rebooting the router does work. But this happens everytime after I do a upgrade. I have tried this 3-4 times already. So does this mean everytime we need to reboot the router after upgrade?
Also why the creation of VMs with the same router succeeds? And why don't we see this error before upgrade ?

                
> Adding VMs to an existing LB rule after upgrading from CS 3.0.2 to ASF 4.0 fails with "ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Unable to apply ip association on router "
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-127
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-127
>             Project: CloudStack
>          Issue Type: Bug
>          Components: Network Controller
>    Affects Versions: pre-4.0.0
>         Environment: MS OS : Rhel 6.2
> Hypervisor : Xen 6.0.2
> BUILDs : 3.0.2 and ASF 4.0 [ Git Revision: 54f9af1695bc359b02e9fc906b3b335cc0bfec41 Git URL: https://git-wip-us.apache.org/repos/asf/incubator-cloudstack.git]
>            Reporter: Abhinav Roy
>            Assignee: Sheng Yang
>             Fix For: pre-4.0.0
>
>         Attachments: management-server.log, messages, SMlog.10, SMlog.11
>
>
> Steps :
> =========================
> 1. Deploy an advanced zone setup with CS 3.0.2
> 2. Deploy couple of VMs.
> 3. Acquire an IP Address for the Network created and create a LB rule for the VMs created in step2.
> 4. Upgrade the setup to ASF 4.0
> 5. Create a VM on the existing Network.
> 6. Add the VM created in step 5 to the LB rule created in step 3.
> Expected Behaviour :
> =========================
> All the steps should be successful and no error should be seen.
> Observed Behaviour :
> =========================
> Step 6 fails with the following exceptions
> 2012-09-18 00:32:07,036 DEBUG [network.lb.LoadBalancingRulesManagerImpl] (Job-Executor-7:job-18) Adding VM[User|VM-3] to the load balancer pool
> 2012-09-18 00:32:07,130 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (Job-Executor-7:job-18) Applying ip association in network Ntwk[204|Guest|7]
> 2012-09-18 00:32:07,177 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Sending  { Cmd , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 100001, [{"routing.IpAssocCommand":{"ipAddresses":[{"accountId":2,"publicIp":"10.102.125.69","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:4d:b6:00:00:09","networkRate":200,"trafficType":"Public"},{"accountId":2,"publicIp":"10.102.125.66","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:b2:8e:00:00:06","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.2.233","router.name":"r-4-VM"},"wait":0}}] }
> 2012-09-18 00:32:07,177 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Executing:  { Cmd , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 100001, [{"routing.IpAssocCommand":{"ipAddresses":[{"accountId":2,"publicIp":"10.102.125.69","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:4d:b6:00:00:09","networkRate":200,"trafficType":"Public"},{"accountId":2,"publicIp":"10.102.125.66","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:b2:8e:00:00:06","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.2.233","router.name":"r-4-VM"},"wait":0}}] }
> 2012-09-18 00:32:07,178 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-6:null) Seq 1-2076573728: Executing request
> 2012-09-18 00:32:07,718 ERROR [xen.resource.CitrixResourceBase] (DirectAgent-6:null) Ip Assoc failure on applying one ip due to exception:  
> com.cloud.exception.InternalErrorException: Xen plugin "ipassoc" failed.
>         at com.cloud.hypervisor.xen.resource.CitrixResourceBase.assignPublicIpAddress(CitrixResourceBase.java:1934)
>         at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:2030)
>         at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:421)
>         at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73)
>         at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:191)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
>         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,719 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-6:null) Seq 1-2076573728: Response Received: 
> 2012-09-18 00:32:07,719 DEBUG [agent.transport.Request] (DirectAgent-6:null) Seq 1-2076573728: Processing:  { Ans: , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 0, [{"routing.IpAssocAnswer":{"results":["Failed",null],"result":false,"wait":0}}] }
> 2012-09-18 00:32:07,720 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Received:  { Ans: , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 0, { IpAssocAnswer } }
> 2012-09-18 00:32:07,723 WARN  [network.lb.LoadBalancingRulesManagerImpl] (Job-Executor-7:job-18) Unable to apply the load balancer config because resource is unavaliable.
> com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Unable to apply ip association on router 
>         at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyRules(VirtualNetworkApplianceManagerImpl.java:3135)
>         at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.associatePublicIP(VirtualNetworkApplianceManagerImpl.java:2983)
>         at com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.associatePublicIP(VpcVirtualNetworkApplianceManagerImpl.java:551)
>         at com.cloud.network.element.VirtualRouterElement.applyIps(VirtualRouterElement.java:435)
>         at com.cloud.network.NetworkManagerImpl.applyIpAssociations(NetworkManagerImpl.java:966)
>         at com.cloud.network.NetworkManagerImpl.applyRules(NetworkManagerImpl.java:3668)
>         at com.cloud.network.lb.LoadBalancingRulesManagerImpl.applyLoadBalancerRules(LoadBalancingRulesManagerImpl.java:922)
>         at com.cloud.utils.db.DatabaseCallback.intercept(DatabaseCallback.java:34)
>         at com.cloud.network.lb.LoadBalancingRulesManagerImpl.applyLoadBalancerConfig(LoadBalancingRulesManagerImpl.java:896)
>         at com.cloud.network.lb.LoadBalancingRulesManagerImpl.assignToLoadBalancer(LoadBalancingRulesManagerImpl.java:467)
>         at com.cloud.utils.component.ComponentLocator$InterceptorDispatcher.intercept(ComponentLocator.java:1231)
>         at com.cloud.api.commands.AssignToLoadBalancerRuleCmd.execute(AssignToLoadBalancerRuleCmd.java:99)
>         at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
>         at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,731 ERROR [cloud.api.ApiDispatcher] (Job-Executor-7:job-18) Exception while executing AssignToLoadBalancerRuleCmd:
> com.cloud.utils.exception.CloudRuntimeException: Failed to add specified loadbalancerruleid for vms [6]
>         at com.cloud.network.lb.LoadBalancingRulesManagerImpl.assignToLoadBalancer(LoadBalancingRulesManagerImpl.java:489)
>         at com.cloud.utils.component.ComponentLocator$InterceptorDispatcher.intercept(ComponentLocator.java:1231)
>         at com.cloud.api.commands.AssignToLoadBalancerRuleCmd.execute(AssignToLoadBalancerRuleCmd.java:99)
>         at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
>         at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,732 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-7:job-18) Complete async job-18, jobStatus: 2, resultCode: 530, result: Error Code: 530 Error text: Failed to add specified loadbalancerruleid for vms [6]
> 2012-09-18 00:32:07,756 DEBUG [cloud.async.SyncQueueManagerImpl] (Job-Executor-7:job-18) Sync queue (1) is currently empty
> 2012-09-18 00:32:08,755 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-8:null) Ping from 1

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira