cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cloud List <cloud-l...@sg.or.id>
Subject Unable to start agent after roll-back to previous ACS version
Date Thu, 20 Oct 2016 10:49:36 GMT
Hi,

We are using ACS version 4.2 / 4.9 on our test environment. We are using
Ubuntu 12.04 as the operating system and KVM as the hypervisor.

We are trying to simulate an upgrade from ACS 4.2 to 4.9 and roll-back from
4.9 to 4.2 on our test environment. The upgrade went smooth, and the
roll-back went well as well except when we need to start the agent after
downgrading the agent.

After uninstalling cloudstack-agent version 4.9 and installing back
cloudstack-agent version 4.2, I am not able to start the agent with below
error messages:

====
2016-10-20 17:32:28,187 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/cloud0/bridge
2016-10-20 17:32:28,187 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) Found bridge cloud0
2016-10-20 17:32:28,188 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/lo/bridge
2016-10-20 17:32:28,188 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file
/sys/devices/virtual/net/cloudbr1/bridge
2016-10-20 17:32:28,188 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) Found bridge cloudbr1
2016-10-20 17:32:28,188 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet4/bridge
2016-10-20 17:32:28,188 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet5/bridge
2016-10-20 17:32:28,188 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet6/bridge
2016-10-20 17:32:28,188 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet7/bridge
2016-10-20 17:32:28,188 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet8/bridge
2016-10-20 17:32:28,189 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet9/bridge
2016-10-20 17:32:28,189 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet0/bridge
2016-10-20 17:32:28,189 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet1/bridge
2016-10-20 17:32:28,189 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet2/bridge
2016-10-20 17:32:28,189 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet3/bridge
2016-10-20 17:32:28,189 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/virbr0/bridge
2016-10-20 17:32:28,189 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) Found bridge virbr0
2016-10-20 17:32:28,189 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet11/bridge
2016-10-20 17:32:28,190 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet12/bridge
2016-10-20 17:32:28,190 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet13/bridge
2016-10-20 17:32:28,190 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet14/bridge
2016-10-20 17:32:28,190 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet10/bridge
2016-10-20 17:32:28,190 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking for pif for bridge cloud0
2016-10-20 17:32:28,190 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) matchPifFileInDirectory: file name 'vnet1'
2016-10-20 17:32:28,191 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) matchPifFileInDirectory: file name 'vnet2'
2016-10-20 17:32:28,191 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) matchPifFileInDirectory: file name 'vnet5'
2016-10-20 17:32:28,191 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) failing to get physical interface from bridge cloud0,
did not find an eth*, bond*, vlan*, em*, or p*p* in
/sys/devices/virtual/net/cloud0/brif
2016-10-20 17:32:28,191 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking for pif for bridge cloudbr1
2016-10-20 17:32:28,191 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) matchPifFileInDirectory: file name 'eth1'
2016-10-20 17:32:28,192 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking for pif for bridge virbr0
2016-10-20 17:32:28,192 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) failing to get physical interface from bridge virbr0,
did not find an eth*, bond*, vlan*, em*, or p*p* in
/sys/devices/virtual/net/virbr0/brif
2016-10-20 17:32:28,192 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) done looking for pifs, no more bridges
2016-10-20 17:32:28,192 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) Failed to get private nic name
2016-10-20 17:32:28,192 ERROR [cloud.agent.AgentShell] (main:null) (logid:)
Unable to start agent: Failed to get private nic name
====

Below is the result of brctl show and the content of
/etc/network/interfaces:

====
root@test-kvm-03:/var/log/cloudstack/agent# brctl show
bridge name     bridge id               STP enabled     interfaces
cloud0          8000.fe00a9fe00f8       no              vnet1
                                                        vnet2
                                                        vnet5
cloudbr1                8000.d067e5ec82c0       no              eth1
                                                        vnet0
                                                        vnet10
                                                        vnet11
                                                        vnet12
                                                        vnet13
                                                        vnet14
                                                        vnet3
                                                        vnet4
                                                        vnet6
                                                        vnet7
                                                        vnet8
                                                        vnet9
virbr0          8000.000000000000       yes
====

/etc/network/interfaces:

====
# The loopback network interface
auto lo
iface lo inet loopback

auto eth1
#iface eth1 inet static
iface eth1 inet manual

auto cloudbr1
iface cloudbr1 inet static
bridge_ports eth1
        address 192.168.0.201
        netmask 255.255.255.0
        network 192.168.0.0
        broadcast 192.168.0.255
        gateway 192.168.0.1
        dns-nameservers 8.8.8.8 8.8.4.4
        dns-search xxxxx.com

auto cloudbr1:0
iface cloudbr1:0 inet static
        address 192.168.3.201
        netmask 255.255.255.0
        network 192.168.3.0
        broadcast 192.168.3.255
====

It seems that the error messages are complaining about no physical
interface being added into cloud0 and virbr0 bridges. I tried to add eth1,
cloudbr1 into the bridges but it didn't work. Deleting the cloud0 and
virbr0 bridges doesn't help either, agent is still complaining about cannot
find "pifs":

====
2016-10-20 18:00:48,331 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/lo/bridge
2016-10-20 18:00:48,332 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file
/sys/devices/virtual/net/cloudbr1/bridge
2016-10-20 18:00:48,332 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) Found bridge cloudbr1
2016-10-20 18:00:48,332 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet4/bridge
2016-10-20 18:00:48,332 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet5/bridge
2016-10-20 18:00:48,332 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet6/bridge
2016-10-20 18:00:48,332 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet7/bridge
2016-10-20 18:00:48,332 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet8/bridge
2016-10-20 18:00:48,332 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet9/bridge
2016-10-20 18:00:48,333 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet0/bridge
2016-10-20 18:00:48,333 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet1/bridge
2016-10-20 18:00:48,333 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet2/bridge
2016-10-20 18:00:48,333 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet3/bridge
2016-10-20 18:00:48,333 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet11/bridge
2016-10-20 18:00:48,333 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet12/bridge
2016-10-20 18:00:48,333 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet13/bridge
2016-10-20 18:00:48,333 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet14/bridge
2016-10-20 18:00:48,334 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet10/bridge
2016-10-20 18:00:48,334 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking for pif for bridge cloudbr1
2016-10-20 18:00:48,334 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) matchPifFileInDirectory: file name 'eth1'
2016-10-20 18:00:48,334 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) done looking for pifs, no more bridges
2016-10-20 18:00:48,334 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) Failed to get private nic name
2016-10-20 18:00:48,334 ERROR [cloud.agent.AgentShell] (main:null) (logid:)
Unable to start agent: Failed to get private nic name
====

I understand that the required bridge information is supposed to be added
by CloudStack during the time when the host is added. Is there a way how I
can add the bridge information again manually without having to delete and
re-add the host into CloudStack? The reason is because we want to keep the
VMs running during the downgrade, deleting and re-adding the host into
CloudStack will shutdown the VMs.

Any advice is greatly appreciated.

Thank you.

-ip-

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message