Mailing-List: contact dev-help@cloudstack.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cloudstack.apache.org
Received-SPF: softfail (athena.apache.org: transitioning domain of
 florin.dumitrascu@intunenetworks.com does not designate 84.203.208.122 as
 permitted sender)
From: Florin Dumitrascu <florin.dumitrascu@intunenetworks.com>
To: "dev@cloudstack.apache.org" <dev@cloudstack.apache.org>
Subject: [CS 4.3 | XenServer] Unable to start Virtual Router
 (com.cloud.exception.AgentUnavailableException)
Thread-Topic: [CS 4.3 | XenServer] Unable to start Virtual Router
 (com.cloud.exception.AgentUnavailableException)
Thread-Index: Ac+Erx4uFlS+pI7vR9SMUePSe+3jzw==
Date: Tue, 10 Jun 2014 13:23:06 +0000
Message-ID: 
 <D016916FFAD2DE4990667CD795FA1F78043325E2@DubExch.intunetech.local>
Accept-Language: en-US, en-IE
Content-Language: en-US
Content-Type: multipart/alternative;
	boundary="_000_D016916FFAD2DE4990667CD795FA1F78043325E2DubExchintunete_"
MIME-Version: 1.0

--_000_D016916FFAD2DE4990667CD795FA1F78043325E2DubExchintunete_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Hi,

Could anyone please help with the following issue I am seeing?
I am running CloudStack 4.3 and XenServer 6.2 (SP1)

I have removed a host which was working normally and re-added it (it is Hos=
t 15 in the log appended).
The communication with the host seems to be established as shown in the GUI=
 (State is "Up").
However, every attempt to start a Virtual Router on that host is failing. T=
he reasons that seem to be related to the issue are (as seen in the appende=
d log):

"Unable to get the template/scripts version of router r-187-VM due to: getD=
omRVersionCmd failed"
"com.cloud.exception.AgentUnavailableException: Resource [Host:15] is unrea=
chable"

How can I further debug this ? How can I check the state of the agent runni=
ng on XenServer ?

Thank you for your help,
Florin

Debug Log
---------------

2014-06-10 14:14:10,549 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-292=
:ctx-fb46119b) Seq 15-1518796837: Response Received:
2014-06-10 14:14:10,551 DEBUG [c.c.a.t.Request] (DirectAgent-292:ctx-fb4611=
9b) Seq 15-1518796837: Processing:  { Ans: , MgmtId: 123561959226, via: 15,=
 Ver: v1, Flags: 10, [{"com.cloud.agent.api.StartAnswer":{
"vm":{"id":187,"name":"r-187-VM","bootloader":"PyGrub","type":"DomainRouter=
","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":134217728,"maxRam":134217=
728,"arch":"x86_64","os":"Debian GNU/Linux 7(32-bit)","boo
tArgs":" template=3DdomP name=3Dr-187-VM eth2ip=3D192.168.18.22 eth2mask=3D=
255.255.252.0 gateway=3D192.168.16.1 eth0ip=3D10.10.2.1 eth0mask=3D255.255.=
255.0 domain=3Dcs2cloud.internal dhcprange=3D10.10.2.1 eth1ip=3D169.254.2.2=
54
eth1mask=3D255.255.0.0 type=3Drouter disable_rp_filter=3Dtrue dns1=3D192.16=
8.20.18 dns2=3D192.168.20.81","rebootOnCrash":false,"enableHA":true,"limitC=
puUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"128a547
57b1466cc","params":{},"uuid":"035c06a8-8b08-46e7-88ef-ac128b1e2aeb","disks=
":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"9c8b=
32ff-8864-4681-acb3-3a76b0c04d91","volumeType":"ROOT","dat
aStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"d92=
42b8f-5342-3225-bda3-8b79202af571","id":19,"poolType":"IscsiLUN","host":"10=
.20.1.12","path":"/iqn.2014-03.local.intune:cloudstorsrv3.
target1/1","port":3260,"url":"IscsiLUN://10.20.1.12//iqn.2014-03.local.intu=
ne:cloudstorsrv3.target1/1/?ROLE=3DPrimary&STOREUUID=3Dd9242b8f-5342-3225-b=
da3-8b79202af571"}},"name":"ROOT-187","size":2621440000,"path"
:"05b0fb51-f725-46e5-8af0-43884bfd6edf","volumeId":228,"vmName":"r-187-VM",=
"accountId":2,"format":"VHD","id":228,"deviceId":0,"hypervisorType":"XenSer=
ver"}},"diskSeq":0,"path":"05b0fb51-f725-46e5-8af0-43884bf
d6edf","type":"ROOT","_details":{"managed":"false","storagePort":"3260","st=
orageHost":"10.20.1.12","volumeSize":"2621440000"}}],"nics":[{"deviceId":2,=
"networkRateMbps":200,"defaultNic":true,"uuid":"52d1b19d-b
053-4358-9659-6462f4d320ba","ip":"192.168.18.22","netmask":"255.255.252.0",=
"gateway":"192.168.16.1","mac":"06:2c:bc:00:00:22","dns1":"192.168.20.18","=
dns2":"192.168.20.81","broadcastType":"Vlan","type":"Publi
c","broadcastUri":"vlan://untagged","isolationUri":"vlan://untagged","isSec=
urityGroupEnabled":false,"name":"PUBLIC"},{"deviceId":0,"networkRateMbps":2=
00,"defaultNic":false,"uuid":"ea1180aa-9889-4db5-8414-6d0b
b9653f65","ip":"10.10.2.1","netmask":"255.255.255.0","mac":"02:00:4a:0a:00:=
02","dns1":"192.168.20.18","dns2":"192.168.20.81","broadcastType":"Vswitch"=
,"type":"Guest","broadcastUri":"vs://385","isolationUri":"
vs://385","isSecurityGroupEnabled":false,"name":"GUEST"},{"deviceId":1,"net=
workRateMbps":-1,"defaultNic":false,"uuid":"c17a414d-e0d1-4ff8-80a3-6db65c5=
af96e","ip":"169.254.2.254","netmask":"255.255.0.0","gatew
ay":"169.254.0.1","mac":"0e:00:a9:fe:02:fe","broadcastType":"LinkLocal","ty=
pe":"Control","isSecurityGroupEnabled":false}]},"_iqnToPath":{},"result":tr=
ue,"wait":0}},{"com.cloud.agent.api.check.CheckSshAnswer":
{"result":true,"wait":0}},{"com.cloud.agent.api.GetDomRVersionAnswer":{"res=
ult":false,"details":"getDomRVersionCmd failed","wait":0}}] }
2014-06-10 14:14:10,551 DEBUG [c.c.a.t.Request] (Job-Executor-17:ctx-7e9e98=
c6 ctx-8fd8ddc7) Seq 15-1518796837: Received:  { Ans: , MgmtId: 12356195922=
6, via: 15, Ver: v1, Flags: 10, { StartAnswer, CheckSshAns
wer, GetDomRVersionAnswer } }
2014-06-10 14:14:10,602 WARN  [c.c.n.r.VirtualNetworkApplianceManagerImpl] =
(Job-Executor-17:ctx-7e9e98c6 ctx-8fd8ddc7) Unable to get the template/scri=
pts version of router r-187-VM due to: getDomRVersionCmd f
ailed
2014-06-10 14:14:10,602 INFO  [c.c.v.VirtualMachineManagerImpl] (Job-Execut=
or-17:ctx-7e9e98c6 ctx-8fd8ddc7) The guru did not like the answers so stopp=
ing VM[DomainRouter|r-187-VM]
2014-06-10 14:14:10,605 DEBUG [c.c.a.t.Request] (Job-Executor-17:ctx-7e9e98=
c6 ctx-8fd8ddc7) Seq 15-1518796840: Sending  { Cmd , MgmtId: 123561959226, =
via: 15(xenserver2), Ver: v1, Flags: 100011, [{"com.cloud.
agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"vmName":=
"r-187-VM","wait":0}}] }
2014-06-10 14:14:10,605 DEBUG [c.c.a.t.Request] (Job-Executor-17:ctx-7e9e98=
c6 ctx-8fd8ddc7) Seq 15-1518796840: Executing:  { Cmd , MgmtId: 12356195922=
6, via: 15(xenserver2), Ver: v1, Flags: 100011, [{"com.clo
ud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"vmNam=
e":"r-187-VM","wait":0}}] }
2014-06-10 14:14:10,606 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-494=
:ctx-2c6d4aa5) Seq 15-1518796840: Executing request
2014-06-10 14:14:10,694 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-4=
94:ctx-2c6d4aa5) 9. The VM r-187-VM is in Stopping state

2014-06-10 14:14:20,772 DEBUG [c.c.a.t.Request] (Job-Executor-17:ctx-7e9e98=
c6 ctx-8fd8ddc7) Seq 15-1518796840: Received:  { Ans: , MgmtId: 12356195922=
6, via: 15, Ver: v1, Flags: 10, { StopAnswer } }
2014-06-10 14:14:20,772 DEBUG [c.c.a.m.AgentManagerImpl] (Job-Executor-17:c=
tx-7e9e98c6 ctx-8fd8ddc7) Details from executing class com.cloud.agent.api.=
StopCommand: Stop VM r-187-VM Succeed
2014-06-10 14:14:20,772 ERROR [c.c.v.VirtualMachineManagerImpl] (Job-Execut=
or-17:ctx-7e9e98c6 ctx-8fd8ddc7) Failed to start instance VM[DomainRouter|r=
-187-VM]
com.cloud.utils.exception.ExecutionException: Unable to start VM[DomainRout=
er|r-187-VM] due to error in finalizeStart, not retrying

2014-06-10 14:14:22,490 INFO  [c.c.v.VirtualMachineManagerImpl] (Job-Execut=
or-17:ctx-7e9e98c6 ctx-8fd8ddc7) Unable to contact resource.
com.cloud.exception.AgentUnavailableException: Resource [Host:15] is unreac=
hable: Host 15: Unable to start instance due to Unable to start VM[DomainRo=
uter|r-187-VM] due to error in finalizeStart, not retrying
        at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualM=
achineManagerImpl.java:1072)
IMPORTANT NOTE: The information in this e-mail (and any attachments) is con=
fidential. The contents may not be disclosed or used by anyone other than t=
he addressee. If you are not the intended recipient, please notify the send=
er immediately or telephone: +353 (0)1 6204700. We cannot accept any respon=
sibility for the accuracy or completeness of this message as it has been tr=
ansmitted over a public network. If you suspect that the message may have b=
een intercepted or amended, please call the sender.

--_000_D016916FFAD2DE4990667CD795FA1F78043325E2DubExchintunete_--