incubator-cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nik Martin <nik.mar...@nfinausa.com>
Subject Cannot use NIC bond on guest/public networks
Date Wed, 05 Sep 2012 18:11:25 GMT
I'm having an issue using bonded interfaces on gguest and public 
networks.  Details:

3 HVs, Xenserver 6.02 all patches applied, open vswitch backend, sixe 
nics in each, configured as:
eth0 management
eth1+3 bonded, tagged as guest_pub
eth2, unused(management backup)
eth4+5 bonded, tagged as storage, which is iSCSI based

Each bond slave connects to  pair of L3 switches that are properly stacked

If I unbond bond 1+3, vms are fine, guest and public traffic flows fine. 
  If I stop cloud-management service, open xencenter and bond eth1+3, 
then name the resulting network "guest_pub", it now carries the label 
for my network tag for guest and public traffic.

When I restart cloud-management, as soon as my system VMs try to start, 
they start bouncing wildly in xencenter.  The VIF is created, and looks 
like "VLAN-SOMEUUID-15, where 15 is the vlan for my public network.  The 
VIF never gets assigned to the bond.  If I look in the cloudstack log, I 
see:

(DirectAgent-97:null) PV args are -- quiet 
console=hvc0%template=domP%type=consoleproxy%host=172.16.5.2%port=8250%name=v-2-VM%premium=true%zone=1%pod=1%guid=Proxy.2%proxy_vm=2%disable_rp_filter=true%eth2ip=172.16.15.118%eth2mask=255.255.255.0%gateway=172.16.15.1%eth0ip=169.254.2.227%eth0mask=255.255.0.0%eth1ip=172.16.5.34%eth1mask=255.255.255.0%mgmtcidr=172.16.5.0/24%localgw=172.16.5.1%internaldns1=8.8.8.8%internaldns2=8.8.4.4%dns1=8.8.8.8%dns2=8.8.4.4
2012-09-05 12:54:31,762 DEBUG [xen.resource.CitrixResourceBase] 
(DirectAgent-97:null) VBD 14e0d8bc-8b5b-2316-cd1d-07c04389e242 created 
for Vol[2|ROOT|a77d8900-aa57-4ce9-9c73-6db35d985f48|2147483648]
2012-09-05 12:54:31,824 DEBUG [xen.resource.CitrixResourceBase] 
(DirectAgent-97:null) Creating VIF for v-2-VM on nic 
[Nic:Public-172.16.15.118-vlan://15]
2012-09-05 12:54:31,824 DEBUG [xen.resource.CitrixResourceBase] 
(DirectAgent-97:null) Looking for network named guest_pub
2012-09-05 12:54:31,829 DEBUG [xen.resource.CitrixResourceBase] 
(DirectAgent-97:null) Found more than one network with the name guest_pub
2012-09-05 12:54:31,845 DEBUG [xen.resource.CitrixResourceBase] 
(DirectAgent-97:null) Found a network called guest_pub on 
host=172.16.5.5;  Network=f008aa20-1ea1-f6e5-c881-08911e3549e5; 
pif=64d27657-d3be-df29-d365-0baa2dab5303
2012-09-05 12:54:31,855 DEBUG [xen.resource.CitrixResourceBase] 
(DirectAgent-97:null) Creating VLAN 15 on host 172.16.5.5 on device eth1
2012-09-05 12:54:31,885 WARN  [xen.resource.CitrixResourceBase] 
(DirectAgent-97:null) Catch Exception: class 
com.xensource.xenapi.Types$XenAPIException due to 
CANNOT_ADD_VLAN_TO_BOND_SLAVEOpaqueRef:70faa00e-7444-ee41-3cf3-7d151b6fd561
CANNOT_ADD_VLAN_TO_BOND_SLAVEOpaqueRef:70faa00e-7444-ee41-3cf3-7d151b6fd561
	at com.xensource.xenapi.Types.checkResponse(Types.java:1731)
	at com.xensource.xenapi.Connection.dispatch(Connection.java:372)
	at 
com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:905)
	at com.xensource.xenapi.VLAN.create(VLAN.java:349)
	at 
com.cloud.hypervisor.xen.resource.CitrixResourceBase.enableVlanNetwork(CitrixResourceBase.java:3646)
	at 
com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNetwork(CitrixResourceBase.java:635)
	at 
com.cloud.hypervisor.xen.resource.CitrixResourceBase.createVif(CitrixResourceBase.java:671)
	at 
com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1104)
	at 
com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:466)
	at 
com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:69)
	at 
com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:187)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
	at java.util.concurrent.FutureTask.run(FutureTask.java:166)
	at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
	at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
	at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
	at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
	at java.lang.Thread.run(Thread.java:679)
2012-09-05 12:54:31,886 WARN  [xen.resource.CitrixResourceBase] 
(DirectAgent-97:null) Unable to start v-2-VM due to
CANNOT_ADD_VLAN_TO_BOND_SLAVEOpaqueRef:70faa00e-7444-ee41-3cf3-7d151b6fd561
	at com.xensource.xenapi.Types.checkResponse(Types.java:1731)
	at com.xensource.xenapi.Connection.dispatch(Connection.java:372)
	at 
com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:905)
	at com.xensource.xenapi.VLAN.create(VLAN.java:349)
	at 
com.cloud.hypervisor.xen.resource.CitrixResourceBase.enableVlanNetwork(CitrixResourceBase.java:3646)
	at 
com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNetwork(CitrixResourceBase.java:635)
	at 
com.cloud.hypervisor.xen.resource.CitrixResourceBase.createVif(CitrixResourceBase.java:671)
	at 
com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1104)
	at 
com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:466)
	at 
com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:69)
	at 
com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:187)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
	at java.util.concurrent.FutureTask.run(FutureTask.java:166)
	at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
	at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
	at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
	at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
	at java.lang.Thread.run(Thread.java:679)
2012-09-05 12:54:31,977 WARN  [xen.resource.CitrixResourceBase] 
(DirectAgent-97:null) Unable to clean up VBD due to
You gave an invalid object reference.  The object may have recently been 
deleted.  The class parameter gives the type of reference given, and the 
handle parameter echoes the bad value given.
	at com.xensource.xenapi.Types.checkResponse(Types.java:211)
	at com.xensource.xenapi.Connection.dispatch(Connection.java:372)
	at 
com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:905)
	at com.xensource.xenapi.VBD.unplug(VBD.java:1058)
	at 
com.cloud.hypervisor.xen.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:929)
	at 
com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1167)
	at 
com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:466)
	at 
com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:69)
	at 
com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:187)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
	at java.util.concurrent.FutureTask.run(FutureTask.java:166)
	at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
	at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
	at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
	at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
	at java.lang.Thread.run(Thread.java:679)
2012-09-05 12:54:31,996 WARN  [xen.resource.CitrixResourceBase] 
(DirectAgent-97:null) Unable to clean up VBD due to
You gave an invalid object reference.  The object may have recently been 
deleted.  The class parameter gives the type of reference given, and the 
handle parameter echoes the bad value given.
	at com.xensource.xenapi.Types.checkResponse(Types.java:211)
	at com.xensource.xenapi.Connection.dispatch(Connection.java:372)
	at 
com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:905)
	at com.xensource.xenapi.VBD.unplug(VBD.java:1058)
	at 
com.cloud.hypervisor.xen.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:929)
	at 
com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1167)
	at 
com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:466)
	at 
com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:69)
	at 
com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:187)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
	at java.util.concurrent.FutureTask.run(FutureTask.java:166)
	at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
	at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
	at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
	at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
	at java.lang.Thread.run(Thread.java:679)
2012-09-05 12:54:31,996 DEBUG [xen.resource.CitrixResourceBase] 
(DirectAgent-97:null) The VM is in stopped state, detected problem 
during startup : v-2-VM
2012-09-05 12:54:31,996 DEBUG [agent.manager.DirectAgentAttache] 
(DirectAgent-97:null) Seq 4-204669124: Cancelling because one of the 
answers is false and it is stop on error.
2012-09-05 12:54:31,996 DEBUG [agent.manager.DirectAgentAttache] 
(DirectAgent-97:null) Seq 4-204669124: Response Received:
2012-09-05 12:54:31,997 DEBUG [agent.transport.Request] 
(DirectAgent-97:null) Seq 4-204669124: Processing:  { Ans: , MgmtId: 
130577622632, via: 4, Ver: v1, Flags: 110, 
[{"StartAnswer":{"vm":{"id":2,"name":"v-2-VM","bootloader":"PyGrub","type":"ConsoleProxy","cpus":1,"speed":500,"minRam":1073741824,"maxRam":1073741824,"arch":"x86_64","os":"Debian

GNU/Linux 5.0 (32-bit)","bootArgs":" template=domP type=consoleproxy 
host=172.16.5.2 port=8250 name=v-2-VM premium=true zone=1 pod=1 
guid=Proxy.2 proxy_vm=2 disable_rp_filter=true eth2ip=172.16.15.118 
eth2mask=255.255.255.0 gateway=172.16.15.1 eth0ip=169.254.2.227 
eth0mask=255.255.0.0 eth1ip=172.16.5.34 eth1mask=255.255.255.0 
mgmtcidr=172.16.5.0/24 localgw=172.16.5.1 internaldns1=8.8.8.8 
internaldns2=8.8.4.4 dns1=8.8.8.8 
dns2=8.8.4.4","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"vncPassword":"9d8805d384dc10ce","params":{},"disks":[{"id":2,"name":"ROOT-2","mountPoint":"/iqn.2012-01.com.nfinausa:san01/1","path":"a77d8900-aa57-4ce9-9c73-6db35d985f48","size":2147483648,"type":"ROOT","storagePoolType":"IscsiLUN","storagePoolUuid":"776299a8-a770-30f7-9f6b-8c422790f5b7","deviceId":0}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"ip":"172.16.15.118","netmask":"255.255.255.0","gateway":"172.16.15.1","mac":"06:18:16:00:00:27","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Public","broadcastUri":"vlan://15","isolationUri":"vlan://15","isSecurityGroupEnabled":false,"name":"guest_pub"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"ip":"169.254.2.227","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:02:e3","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":f
 alse,"ip"
:"172.16.5.34","netmask":"255.255.255.0","gateway":"172.16.5.1","mac":"06:58:58:00:00:0f","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"management"}]},"result":false,"details":"Unable

to start v-2-VM due to ","wait":0}}] }
2012-09-05 12:54:31,997 WARN  [cloud.vm.VirtualMachineManagerImpl] 
(DirectAgent-97:null) Cleanup failed due to Unable to start v-2-VM due to
2012-09-05 12:54:31,997 DEBUG [agent.transport.Request] 
(consoleproxy-1:null) Seq 4-204669124: Received:  { Ans: , MgmtId: 
130577622632, via: 4, Ver: v1, Flags: 110, { StartAnswer } }
2012-09-05 12:54:31,997 DEBUG [agent.manager.AgentAttache] 
(DirectAgent-97:null) Seq 4-204669124: No more commands found
2012-09-05 12:54:31,997 WARN  [cloud.vm.VirtualMachineManagerImpl] 
(consoleproxy-1:null) Cleanup failed due to Unable to start v-2-VM due to
2012-09-05 12:54:32,002 INFO  [cloud.vm.VirtualMachineManagerImpl] 
(consoleproxy-1:null) Unable to start VM on Host[-4-Routing] due to 
Unable to start v-2-VM due to
2012-09-05 12:54:32,005 DEBUG [cloud.vm.VirtualMachineManagerImpl] 
(consoleproxy-1:null) Cleaning up resources for the vm 
VM[ConsoleProxy|v-2-VM] in Starting state
2012-09-05 12:54:32,006 DEBUG [agent.transport.Request] 
(consoleproxy-1:null) Seq 4-204669125: Sending  { Cmd , MgmtId: 
130577622632, via: 4, Ver: v1, Flags: 100111, 
[{"StopCommand":{"isProxy":false,"vmName":"v-2-VM","wait":0}}] }
2012-09-05 12:54:32,006 DEBUG [agent.transport.Request] 
(consoleproxy-1:null) Seq 4-204669125: Executing:  { Cmd , MgmtId: 
130577622632, via: 4, Ver: v1, Flags: 100111, 
[{"StopCommand":{"isProxy":false,"vmName":"v-2-VM","wait":0}}] }
2012-09-05 12:54:32,006 DEBUG [agent.manager.DirectAgentAttache] 
(DirectAgent-104:null) Seq 4-204669125: Executing request
2012-09-05 12:54:32,097 INFO  [xen.resource.CitrixResourceBase] 
(DirectAgent-104:null) VM does not exist on 
XenServer0c091805-ebac-4279-9899-52018163a557
2012-09-05 12:54:32,097 DEBUG [agent.manager.DirectAgentAttache] 
(DirectAgent-104:null) Seq 4-204669125: Response Received:
2012-09-05 12:54:32,098 DEBUG [agent.transport.Request] 
(DirectAgent-104:null) Seq 4-204669125: Processing:  { Ans: , MgmtId: 
130577622632, via: 4, Ver: v1, Flags: 110, 
[{"StopAnswer":{"vncPort":0,"bytesSent":0,"bytesReceived":0,"result":true,"details":"VM 
does not exist","wait":0}}] }
2012-09-05 12:54:32,098 DEBUG [cloud.vm.VirtualMachineManagerImpl] 
(DirectAgent-104:null) Cleanup succeeded. Details VM does not exist
2012-09-05 12:54:32,098 DEBUG [agent.transport.Request] 
(consoleproxy-1:null) Seq 4-204669125: Received:  { Ans: , MgmtId: 
130577622632, via: 4, Ver: v1, Flags: 110, { StopAnswer } }
2012-09-05 12:54:32,098 DEBUG [agent.manager.AgentAttache] 
(DirectAgent-104:null) Seq 4-204669125: No more commands found
2012-09-05 12:54:32,098 DEBUG [cloud.vm.VirtualMachineManagerImpl] 
(consoleproxy-1:null) Cleanup succeeded. Details VM does not exist
2012-09-05 12:54:32,119 DEBUG [dc.dao.DataCenterIpAddressDaoImpl] 
(consoleproxy-1:null) Releasing ip address for 
reservationId=636f327b-5183-46bd-adf2-57bf0d5fae80, instance=7
2012-09-05 12:54:32,125 DEBUG [cloud.vm.VirtualMachineManagerImpl] 
(consoleproxy-1:null) Successfully cleanued up resources for the vm 
VM[ConsoleProxy|v-2-VM] in Starting state
2012-09-05 12:54:32,126 DEBUG [cloud.vm.VirtualMachineManagerImpl] 
(consoleproxy-1:null) Root volume is ready, need to place VM in volume's 
cluster
2012-09-05 12:54:32,126 DEBUG [cloud.vm.VirtualMachineManagerImpl] 
(consoleproxy-1:null) Vol[2|vm=2|ROOT] is READY, changing deployment 
plan to use this pool's dcId: 1 , podId: 1 , and clusterId: 1
2012-09-05 12:54:32,127 DEBUG [cloud.deploy.FirstFitPlanner] 
(consoleproxy-1:null) DeploymentPlanner allocation algorithm: random
2012-09-05 12:54:32,127 DEBUG [cloud.deploy.FirstFitPlanner] 
(consoleproxy-1:null) Trying to allocate a host and storage pools from 
dc:1, pod:1,cluster:1, requested cpu: 500, requested ram: 1073741824



-- 
Regards,

Nik

Nik Martin
VP Business Development
Nfina Technologies, Inc.
+1.251.243.0043 x1003
Relentless Reliability

Mime
View raw message