Mailing-List: contact issues-help@cloudstack.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cloudstack.apache.org
Date: Wed, 20 Nov 2013 23:15:35 +0000 (UTC)
From: "edison su (JIRA)" <jira@apache.org>
To: cloudstack-issues@incubator.apache.org
Message-ID: <JIRA.12677672.1383692831124.4550.1384989335873@arcas>
In-Reply-To: <JIRA.12677672.1383692831124@arcas>
References: <JIRA.12677672.1383692831124@arcas>
Subject: [jira] [Commented] (CLOUDSTACK-5047) [Automation] KVM - Failed to
 add ACLlist to vpc network,  observed NPE in agent
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/CLOUDSTACK-5047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13828267#comment-13828267 ] 

edison su commented on CLOUDSTACK-5047:
---------------------------------------

I added some logs in commit e03088a3f124407349cffb3ffef5a5f4dfd9c9b4, and fix the NPE issue.

> [Automation] KVM - Failed to add ACLlist to vpc network,  observed NPE in agent 
> --------------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-5047
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5047
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: KVM, Network Controller
>    Affects Versions: 4.3.0
>         Environment: KVM 
> Build : 4.3.0 master
>            Reporter: Rayees Namathponnan
>            Priority: Blocker
>             Fix For: 4.3.0
>
>         Attachments: CLOUDSTACK-5047.rar, libvirtd.log
>
>
> BVT test case integration.smoke.test_privategw_acl.TestPrivateGwACL.test_privategw_acl
> failed with master build 
> observed below error in MS log, while adding ACL list
> 2013-11-05 14:50:30,265 DEBUG [c.c.v.UserVmManagerImpl] (AccountChecker-1:ctx-c25f1452) Firewall rules are removed successfully as a part of vm id=6 expunge
> 2013-11-05 14:50:30,267 DEBUG [c.c.n.r.RulesManagerImpl] (AccountChecker-1:ctx-c25f1452) No port forwarding rules are found for vm id=6
> 2013-11-05 14:50:30,267 DEBUG [c.c.v.UserVmManagerImpl] (AccountChecker-1:ctx-c25f1452) Port forwarding rules are removed successfully as a part of vm id=6 exp
> unge
> 2013-11-05 14:50:30,271 DEBUG [c.c.n.l.LoadBalancingRulesManagerImpl] (AccountChecker-1:ctx-c25f1452) Set load balancer rule for revoke: rule id 49, vmId 6
> 2013-11-05 14:50:30,281 DEBUG [c.c.n.l.LoadBalancingRulesManagerImpl] (AccountChecker-1:ctx-c25f1452) Set load balancer rule for revoke: rule id 49, vmId 6
> 2013-11-05 14:50:30,288 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-0e021cd8) Found 2 routers to update status.
> 2013-11-05 14:50:30,289 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-0e021cd8) Found 0 networks to update RvR status.
> 2013-11-05 14:50:30,316 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (AccountChecker-1:ctx-c25f1452) Applying ip association in network Ntwk[208|Guest|8]
> 2013-11-05 14:50:30,332 DEBUG [c.c.a.t.Request] (AccountChecker-1:ctx-c25f1452) Seq 1-193922916: Sending  { Cmd , MgmtId: 29066118877352, via: 1(Rack2Host11.la
> b.vmops.com), Ver: v1, Flags: 100001, [{"com.cloud.agent.api.routing.IpAssocCommand":{"ipAddresses":[{"accountId":6,"publicIp":"10.223.122.68","sourceNat":true
> ,"add":true,"oneToOneNat":false,"firstIP":true,"broadcastUri":"vlan://1221","vlanGateway":"10.223.122.65","vlanNetmask":"255.255.255.192","vifMacAddress":"06:9
> 6:6a:00:00:3d","networkRate":200,"trafficType":"Public"},{"accountId":6,"publicIp":"10.223.122.76","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":f
> alse,"broadcastUri":"vlan://1221","vlanGateway":"10.223.122.65","vlanNetmask":"255.255.255.192","vifMacAddress":"06:c2:af:00:00:3d","networkRate":200,"trafficT
> ype":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.3.181","router.name":"r-8-QA"},"wait":0}}] }
> 2013-11-05 14:50:30,627 DEBUG [c.c.a.t.Request] (AgentManager-Handler-11:null) Seq 1-193922916: Processing:  { Ans: , MgmtId: 29066118877352, via: 1, Ver: v1,
> Flags: 0, [{"com.cloud.agent.api.Answer":{"result":false,"details":"java.lang.NullPointerException\n\tat com.cloud.agent.api.routing.IpAssocAnswer.<init>(IpAss
> ocAnswer.java:34)\n\tat com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:2062)\n\tat com.cloud.hypervisor.kvm.r
> esource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1278)\n\tat com.cloud.agent.Agent.processRequest(Agent.java:498)\n\tat com.cloud.
> agent.Agent$AgentRequestHandler.doTask(Agent.java:806)\n\tat com.cloud.utils.nio.Task.run(Task.java:83)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker
> (ThreadPoolExecutor.java:1110)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)\n\tat java.lang.Thread.run(Thread.java:679
> )\n","wait":0}}] }
> 2013-11-05 14:50:30,627 DEBUG [c.c.a.t.Request] (AccountChecker-1:ctx-c25f1452) Seq 1-193922916: Received:  { Ans: , MgmtId: 29066118877352, via: 1, Ver: v1, F
> lags: 0, { Answer } }
> 2013-11-05 14:50:30,627 WARN  [c.c.n.l.LoadBalancingRulesManagerImpl] (AccountChecker-1:ctx-c25f1452) Unable to apply the load balancer config because resource
>  is unavaliable.
> com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Unable to apply ip association on router
>         at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyRules(VirtualNetworkApplianceManagerImpl.java:3797)
>         at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.associatePublicIP(VirtualNetworkApplianceManagerImpl.java:3593)
>         at com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.associatePublicIP(VpcVirtualNetworkApplianceManagerImpl.java:505)
>         at sun.reflect.GeneratedMethodAccessor391.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:616)
>         at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
>         at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
> :
> Observed below NPE in KVM agent 
> /primary -m /mnt/fff90cb5-06dd-33b3-8815-d78c08ca01d9 -h 10.223.50.66
> 2013-11-05 14:51:30,323 DEBUG [kvm.resource.KVMHAMonitor] (Thread-315:null) Execution is successful.
> 2013-11-05 14:51:30,325 DEBUG [kvm.resource.KVMHAMonitor] (Thread-315:null) Found NFS storage pool 41b632b5-40b3-3024-a38b-ea259c72579f in libvirt, continuing
> 2013-11-05 14:51:30,325 DEBUG [kvm.resource.KVMHAMonitor] (Thread-315:null) Executing: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh -i 10.223.110.232 -p /export/home/rayees/SC_QA_AUTO4/primary2 -m /mnt/41b632b5-40b3-3024-a38b-ea259c72579f -h 10.223.50.66
> 2013-11-05 14:51:30,344 DEBUG [kvm.resource.KVMHAMonitor] (Thread-315:null) Execution is successful.
> 2013-11-05 14:51:30,391 DEBUG [kvm.resource.BridgeVifDriver] (agentRequest-Handler-4:null) Execution is successful.
> 2013-11-05 14:51:30,391 DEBUG [kvm.resource.BridgeVifDriver] (agentRequest-Handler-4:null) Set name-type for VLAN subsystem. Should be visible in /proc/net/vlan/config
> 2013-11-05 14:51:30,522 WARN  [cloud.agent.Agent] (agentRequest-Handler-4:null) Caught:
> java.lang.NullPointerException
>         at com.cloud.agent.api.routing.IpAssocAnswer.<init>(IpAssocAnswer.java:34)
>         at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:2062)
>         at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1278)
>         at com.cloud.agent.Agent.processRequest(Agent.java:498)
>         at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:806)
>         at com.cloud.utils.nio.Task.run(Task.java:83)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:679)
> 2013-11-05 14:51:30,522 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:null) Seq 1-193922921:  { Ans: , MgmtId: 29066118877352, via: 1, Ver: v1, Flags: 0, [{"com.cloud.agent.api.Answer":{"result":false,"details":"java.lang.NullPointerException\n\tat com.cloud.agent.api.routing.IpAssocAnswer.<init>(IpAssocAnswer.java:34)\n\tat com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:2062)\n\tat com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1278)\n\tat com.cloud.agent.Agent.processRequest(Agent.java:498)\n\tat com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:806)\n\tat com.cloud.utils.nio.Task.run(Task.java:83)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)\n\tat java.lang.Thread.run(Thread.java:679)\n","wait":0}}] }
> 2013-11-05 14:51:35,077 DEBUG [kvm.resource.LibvirtComputingResource] (UgentTask-5:null) Executing: /usr/share/cloudstack-common/scripts/vm/network/security_group.py get_rule_logs_for_vms
> 2013-11-05 14:51:35,621 DEBUG [kvm.resource.LibvirtComputingResource] (UgentTask-5:null) Execution is successful.
> 2013-11-05 14:51:35,621 DEBUG [cloud.agent.Agent] (UgentTask-5:null) Sending ping: Seq 1-308:  { Cmd , MgmtId: -1, via: 1, Ver: v1, Flags: 11, [{"com.cloud.agent.api.PingRoutingWithNwGroupsCommand":{"newGroupStates":{},"newStates":{},"_gatewayAccessible":true,"_vnetAccessible":true,"hostType":"Routing","hostId":1,"wait":0}}] }
> 2013-11-05 14:51:35,627 DEBUG [cloud.agent.Agent] (Agent-Handler-4:null) Received response: Seq 1-308:  { Ans: , MgmtId: 29066118877352, via: 1, Ver: v1, Flags: 100010, [{"com.cloud.agent.api.PingAnswer":{"_command":{"hostType":"Routing","hostId":1,"wait":0},"result":true,"wait":0}}] }
> (END)


--
This message was sent by Atlassian JIRA
(v6.1#6144)