Mailing-List: contact users-help@cloudstack.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@cloudstack.apache.org
Received-SPF: pass (athena.apache.org: domain of andrei@arhont.com designates
 178.248.108.132 as permitted sender)
Date: Mon, 16 Mar 2015 16:11:07 +0000 (GMT)
From: Andrei Mikhailovsky <andrei@arhont.com>
To: users@cloudstack.apache.org
Message-ID: <22910516.6134.1426522262889.JavaMail.andrei@tuchka>
In-Reply-To: <4898963.6082.1426520960097.JavaMail.andrei@tuchka>
Subject: Unable to restart some networks / virtual routers
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_6133_5647340.1426522262887"
Thread-Topic: Unable to restart some networks / virtual routers
Thread-Index: O3IeIGRQ0+/tXXlouu72ltTiKBwwiw==

------=_Part_6133_5647340.1426522262887
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

Hello guys,=20

I am having an issue with restarting couple of networks and their correspon=
ding virtual routers. The majority of networks can be restarted apart from =
a few.=20

I am running ACS 4.4.2 with KVM hypervisors all running on Ubuntu 14.04.=20

Here is the log from the management server:=20


2015-03-16 15:34:01,343 DEBUG [c.c.a.t.Request] (AgentManager-Handler-7:nul=
l) Seq 8-4763401030874431513: Processing: { Ans: , MgmtId: 115129173025118,=
 via: 8, Ver: v1, Flags: 110, [{"com.cloud.agent.api.Answer":{"result":fals=
e,"details":"java.lang.NullPointerException\n\tat com.cloud.network.Network=
s$BroadcastDomainType.getSchemeValue(Networks.java:159)\n\tat com.cloud.net=
work.Networks$BroadcastDomainType.getValue(Networks.java:213)\n\tat com.clo=
ud.hypervisor.kvm.resource.BridgeVifDriver.plug(BridgeVifDriver.java:96)\n\=
tat com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.createVif(Li=
bvirtComputingResource.java:4010)\n\tat com.cloud.hypervisor.kvm.resource.L=
ibvirtComputingResource.createVifs(LibvirtComputingResource.java:3766)\n\ta=
t com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(Libvir=
tComputingResource.java:3795)\n\tat com.cloud.hypervisor.kvm.resource.Libvi=
rtComputingResource.executeRequest(LibvirtComputingResource.java:1333)\n\ta=
t com.cloud.agent.Agent.processRequest(Agent.java:501)\n\tat com.cloud.agen=
t.Agent$AgentRequestHandler.doTask(Agent.java:808)\n\tat com.cloud.utils.ni=
o.Task.run(Task.java:84)\n\tat java.util.concurrent.ThreadPoolExecutor.runW=
orker(ThreadPoolExecutor.java:1145)\n\tat java.util.concurrent.ThreadPoolEx=
ecutor$Worker.run(ThreadPoolExecutor.java:615)\n\tat java.lang.Thread.run(T=
hread.java:745)\n","wait":0}},{"com.cloud.agent.api.Answer":{"result":false=
,"details":"Stopped by previous failure","wait":0}},{"com.cloud.agent.api.A=
nswer":{"result":false,"details":"Stopped by previous failure","wait":0}},{=
"com.cloud.agent.api.Answer":{"result":false,"details":"Stopped by previous=
 failure","wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details=
":"Stopped by previous failure","wait":0}},{"com.cloud.agent.api.Answer":{"=
result":false,"details":"Stopped by previous failure","wait":0}},{"com.clou=
d.agent.api.Answer":{"result":false,"details":"Stopped by previous failure"=
,"wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"Stoppe=
d by previous failure","wait":0}},{"com.cloud.agent.api.Answer":{"result":f=
alse,"details":"Stopped by previous failure","wait":0}},{"com.cloud.agent.a=
pi.Answer":{"result":false,"details":"Stopped by previous failure","wait":0=
}},{"com.cloud.agent.api.Answer":{"result":false,"details":"Stopped by prev=
ious failure","wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"det=
ails":"Stopped by previous failure","wait":0}},{"com.cloud.agent.api.Answer=
":{"result":false,"details":"Stopped by previous failure","wait":0}},{"com.=
cloud.agent.api.Answer":{"result":false,"details":"Stopped by previous fail=
ure","wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"St=
opped by previous failure","wait":0}}] }=20
2015-03-16 15:34:01,343 DEBUG [c.c.a.m.AgentAttache] (AgentManager-Handler-=
7:null) Seq 8-4763401030874431513: No more commands found=20
2015-03-16 15:34:01,344 DEBUG [c.c.a.t.Request] (Work-Job-Executor-7:ctx-4f=
ca30aa job-13975/job-13978 ctx-0d61e6c8) Seq 8-4763401030874431513: Receive=
d: { Ans: , MgmtId: 115129173025118, via: 8, Ver: v1, Flags: 110, { Answer,=
 Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, An=
swer, Answer, Answer, Answer, Answer } }=20
2015-03-16 15:34:01,350 ERROR [c.c.v.VirtualMachineManagerImpl] (Work-Job-E=
xecutor-7:ctx-4fca30aa job-13975/job-13978 ctx-0d61e6c8) Failed to start in=
stance VM[DomainRouter|r-1192-VM]=20
com.cloud.utils.exception.CloudRuntimeException: Unable to get answer that =
is of class com.cloud.agent.api.StartAnswer=20
at com.cloud.agent.manager.Commands.getAnswer(Commands.java:80)=20
at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineMa=
nagerImpl.java:1020)=20
at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineMa=
nagerImpl.java:5195)=20
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)=20
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav=
a:57)=20
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor=
Impl.java:43)=20
at java.lang.reflect.Method.invoke(Method.java:606)=20
at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy=
.java:107)=20
at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineMan=
agerImpl.java:5340)=20
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)=20
at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInCon=
text(AsyncJobManagerImpl.java:503)=20
at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(Manag=
edContextRunnable.java:49)=20
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(=
DefaultManagedContext.java:56)=20
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWit=
hContext(DefaultManagedContext.java:103)=20
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWith=
Context(DefaultManagedContext.java:53)=20
at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(Managed=
ContextRunnable.java:46)=20
at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(Asyn=
cJobManagerImpl.java:460)=20
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)=
=20
at java.util.concurrent.FutureTask.run(FutureTask.java:262)=20
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav=
a:1145)=20
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja=
va:615)=20
at java.lang.Thread.run(Thread.java:745)=20
2015-03-16 15:34:01,354 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-E=
xecutor-7:ctx-4fca30aa job-13975/job-13978 ctx-0d61e6c8) Cleaning up resour=
ces for the vm VM[DomainRouter|r-1192-VM] in Starting state=20


The host server have the following logs (please note that the timestamps sh=
ow different times as they are from two different start attempts, but the e=
ntries are the same every time I attempt to start the virtual router):=20

2015-03-16 16:03:36,781 WARN [cloud.agent.Agent] (agentRequest-Handler-2:nu=
ll) Caught:=20
java.lang.NullPointerException=20
at com.cloud.network.Networks$BroadcastDomainType.getSchemeValue(Networks.j=
ava:159)=20
at com.cloud.network.Networks$BroadcastDomainType.getValue(Networks.java:21=
3)=20
at com.cloud.hypervisor.kvm.resource.BridgeVifDriver.plug(BridgeVifDriver.j=
ava:96)=20
at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.createVif(Lib=
virtComputingResource.java:4010)=20
at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.createVifs(Li=
bvirtComputingResource.java:3766)=20
at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(Libvi=
rtComputingResource.java:3795)=20
at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeReques=
t(LibvirtComputingResource.java:1333)=20
at com.cloud.agent.Agent.processRequest(Agent.java:501)=20
at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:808)=20
at com.cloud.utils.nio.Task.run(Task.java:84)=20
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav=
a:1145)=20
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja=
va:615)=20
at java.lang.Thread.run(Thread.java:745)=20
2015-03-16 16:03:36,784 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:n=
ull) Seq 8-4763401030874431681: { Ans: , MgmtId: 115129173025118, via: 8, V=
er: v1, Flags: 110, [{"com.cloud.agent.api.Answer":{"result":false,"details=
":"java.lang.NullPointerException\n\tat com.cloud.network.Networks$Broadcas=
tDomainType.getSchemeValue(Networks.java:159)\n\tat com.cloud.network.Netwo=
rks$BroadcastDomainType.getValue(Networks.java:213)\n\tat com.cloud.hypervi=
sor.kvm.resource.BridgeVifDriver.plug(BridgeVifDriver.java:96)\n\tat com.cl=
oud.hypervisor.kvm.resource.LibvirtComputingResource.createVif(LibvirtCompu=
tingResource.java:4010)\n\tat com.cloud.hypervisor.kvm.resource.LibvirtComp=
utingResource.createVifs(LibvirtComputingResource.java:3766)\n\tat com.clou=
d.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputing=
Resource.java:3795)\n\tat com.cloud.hypervisor.kvm.resource.LibvirtComputin=
gResource.executeRequest(LibvirtComputingResource.java:1333)\n\tat com.clou=
d.agent.Agent.processRequest(Agent.java:501)\n\tat com.cloud.agent.Agent$Ag=
entRequestHandler.doTask(Agent.java:808)\n\tat com.cloud.utils.nio.Task.run=
(Task.java:84)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(Thre=
adPoolExecutor.java:1145)\n\tat java.util.concurrent.ThreadPoolExecutor$Wor=
ker.run(ThreadPoolExecutor.java:615)\n\tat java.lang.Thread.run(Thread.java=
:745)\n","wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details"=
:"Stopped by previous failure","wait":0}},{"com.cloud.agent.api.Answer":{"r=
esult":false,"details":"Stopped by previous failure","wait":0}},{"com.cloud=
.agent.api.Answer":{"result":false,"details":"Stopped by previous failure",=
"wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"Stopped=
 by previous failure","wait":0}},{"com.cloud.agent.api.Answer":{"result":fa=
lse,"details":"Stopped by previous failure","wait":0}},{"com.cloud.agent.ap=
i.Answer":{"result":false,"details":"Stopped by previous failure","wait":0}=
},{"com.cloud.agent.api.Answer":{"result":false,"details":"Stopped by previ=
ous failure","wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"deta=
ils":"Stopped by previous failure","wait":0}},{"com.cloud.agent.api.Answer"=
:{"result":false,"details":"Stopped by previous failure","wait":0}},{"com.c=
loud.agent.api.Answer":{"result":false,"details":"Stopped by previous failu=
re","wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"Sto=
pped by previous failure","wait":0}},{"com.cloud.agent.api.Answer":{"result=
":false,"details":"Stopped by previous failure","wait":0}},{"com.cloud.agen=
t.api.Answer":{"result":false,"details":"Stopped by previous failure","wait=
":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"Stopped by p=
revious failure","wait":0}}] }=20


Does anyone have any idea what's the problem?=20

Thanks=20

Andrei=20


------=_Part_6133_5647340.1426522262887--