Mailing-List: contact issues-help@cloudstack.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cloudstack.apache.org
Date: Tue, 21 May 2013 04:51:16 +0000 (UTC)
From: "venkata swamybabu budumuru (JIRA)" <jira@apache.org>
To: cloudstack-issues@incubator.apache.org
Message-ID: <JIRA.12648553.1369111607679.351209.1369111876691@arcas>
In-Reply-To: <JIRA.12648553.1369111607679@arcas>
References: <JIRA.12648553.1369111607679@arcas>
Subject: [jira] [Updated] (CLOUDSTACK-2597) [KVM] [SystemVMTemplate]
 java.lang.NullPointerException while bringing up RouterVM for the first
 time
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/CLOUDSTACK-2597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

venkata swamybabu budumuru updated CLOUDSTACK-2597:
---------------------------------------------------

    Attachment: logs.tgz
    
> [KVM] [SystemVMTemplate] java.lang.NullPointerException while bringing up RouterVM for the first time
> -----------------------------------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-2597
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-2597
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: Network Controller
>    Affects Versions: 4.2.0
>         Environment: commit # 85d54cd1c088997dd08f0328984bee1a55703636
>            Reporter: venkata swamybabu budumuru
>             Fix For: 4.2.0
>
>         Attachments: logs.tgz
>
>
> Steps to reproduce :
> 1. Have latest CloudStack build with at least one advanced zone which uses KVM cluster with single host
> 2. Have at least one non-ROOT domain user
> 3. login as above non-ROOT domain user and deploy a VM using an ISOLATED network.
> Observations :
> (i) Above deployVM cmd failed and found the router that was resulted due the vm deployment is in stopped state.
> (ii) Found NPE in agent logs. Here is the snippet of the NPE
> 013-05-21 03:51:31,056 WARN  [resource.virtualnetwork.VirtualRoutingResource] (agentRequest-Handler-1:null) Exception: null
> java.lang.NullPointerException
>         at java.lang.ProcessBuilder.start(ProcessBuilder.java:457)
>         at com.cloud.utils.script.Script.execute(Script.java:183)
>         at com.cloud.utils.script.Script.execute(Script.java:161)
>         at com.cloud.agent.resource.virtualnetwork.VirtualRoutingResource.execute(VirtualRoutingResource.java:651)
>         at com.cloud.agent.resource.virtualnetwork.VirtualRoutingResource.executeRequest(VirtualRoutingResource.java:151)
>         at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1203)
>         at com.cloud.agent.Agent.processRequest(Agent.java:525)
>         at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852)
>         at com.cloud.utils.nio.Task.run(Task.java:83)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:679)
> 2013-05-21 03:53:31,377 WARN  [resource.virtualnetwork.VirtualRoutingResource] (Script-5:null) Interrupting script.
> 2013-05-21 03:53:31,379 WARN  [resource.virtualnetwork.VirtualRoutingResource] (agentRequest-Handler-3:null) Timed out: /usr/share/cloudstack-common/scripts/network/domr/router_proxy.sh ipassoc.sh 169.254.2.189 -A -s -f -l 10.147.44.82/24 -c eth2 -g 10.147.44.1 .  Output is: RTNETLINK answers: File exists
> 2013-05-21 03:54:11,185 WARN  [resource.virtualnetwork.VirtualRoutingResource] (agentRequest-Handler-3:null) Exception: null
> java.lang.NullPointerException
>         at java.lang.ProcessBuilder.start(ProcessBuilder.java:457)
>         at com.cloud.utils.script.Script.execute(Script.java:183)
>         at com.cloud.utils.script.Script.execute(Script.java:161)
>         at com.cloud.agent.resource.virtualnetwork.VirtualRoutingResource.execute(VirtualRoutingResource.java:651)
>         at com.cloud.agent.resource.virtualnetwork.VirtualRoutingResource.executeRequest(VirtualRoutingResource.java:151)
>         at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1203)
>         at com.cloud.agent.Agent.processRequest(Agent.java:525)
>         at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852)
>         at com.cloud.utils.nio.Task.run(Task.java:83)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:679)
> 2013-05-21 03:56:11,471 WARN  [resource.virtualnetwork.VirtualRoutingResource] (Script-6:null) Interrupting script.
> 2013-05-21 03:56:11,472 WARN  [resource.virtualnetwork.VirtualRoutingResource] (agentRequest-Handler-1:null) Timed out: /usr/share/cloudstack-common/scripts/network/domr/router_proxy.sh ipassoc.sh 169.254.1.1 -A -s -f -l 10.147.44.82/24 -c eth2 -g 10.147.44.1 .  Output is: RTNETLINK answers: File exists
> (iii) Tried to deploy another VM in the same above network then everything went fine but, with one NPE in agent logs
> 2013-05-21 04:25:42,682 DEBUG [resource.virtualnetwork.VirtualRoutingResource] (agentRequest-Handler-4:null) Executing: /usr/share/cloudstack-common/scripts/network/domr/router_proxy.sh ipassoc.sh 169.254.3.17 -A -s -f -l 10.147.44.82/24 -c eth2 -g 10.147.44.1
> 2013-05-21 04:25:44,992 DEBUG [resource.virtualnetwork.VirtualRoutingResource] (agentRequest-Handler-4:null) Execution is successful.
> 2013-05-21 04:25:44,993 DEBUG [resource.virtualnetwork.VirtualRoutingResource] (agentRequest-Handler-4:null) RTNETLINK answers: File exists
> iptables: No chain/target/match by that name.
> ARPING 10.147.44.82 from 10.147.44.82 eth2
> Sent 3 probes (3 broadcast(s))
> Received 0 response(s)
> 2013-05-21 04:25:44,993 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:null) Processing command: com.cloud.agent.api.routing.DeleteIpAliasCommand
> 2013-05-21 04:25:44,993 DEBUG [resource.virtualnetwork.VirtualRoutingResource] (agentRequest-Handler-4:null) Executing: null
> 2013-05-21 04:25:44,993 WARN  [resource.virtualnetwork.VirtualRoutingResource] (agentRequest-Handler-4:null) Exception: null
> java.lang.NullPointerException
>         at java.lang.ProcessBuilder.start(ProcessBuilder.java:457)
>         at com.cloud.utils.script.Script.execute(Script.java:183)
>         at com.cloud.utils.script.Script.execute(Script.java:161)
>         at com.cloud.agent.resource.virtualnetwork.VirtualRoutingResource.execute(VirtualRoutingResource.java:651)
>         at com.cloud.agent.resource.virtualnetwork.VirtualRoutingResource.executeRequest(VirtualRoutingResource.java:151)
>         at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1203)
>         at com.cloud.agent.Agent.processRequest(Agent.java:525)
>         at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852)
>         at com.cloud.utils.nio.Task.run(Task.java:83)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:679)
> (iv) Created another non-ROOT domain user and tried to deploy a new VM using ISOLATED network and it went fine without any issues.
> (v) From the above observations, I think the issue happens only on first router VM deployment and any successive attempts go fine without any issues.
> Attaching all the required logs along with db dump to the bug.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira