cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tutkowski, Mike" <Mike.Tutkow...@netapp.com>
Subject Re: VMware VR stuck in Starting state (master branch)
Date Wed, 02 Aug 2017 22:28:39 GMT
However, the workaround that Paul Angus noted in the ticket worked for me just fine.

From: "Tutkowski, Mike" <Mike.Tutkowski@netapp.com>
Date: Wednesday, August 2, 2017 at 4:26 PM
To: "dev@cloudstack.apache.org" <dev@cloudstack.apache.org>
Subject: Re: VMware VR stuck in Starting state (master branch)

I knew this issue sounded familiar. :) I logged a bug for it a couple years ago:

https://issues.apache.org/jira/browse/CLOUDSTACK-9144

From: "Tutkowski, Mike" <Mike.Tutkowski@netapp.com>
Date: Tuesday, August 1, 2017 at 10:22 PM
To: "dev@cloudstack.apache.org" <dev@cloudstack.apache.org>
Subject: VMware VR stuck in Starting state (master branch)

Hi,

I’m running some tests on master with VMware 5.5 in a Basic Zone. My SS and CP VMs come
up fine, but the VR is stuck in the Starting state (even when I manually reboot the VM hosting
the VR).

Anyone have thoughts on this based on the cloud.log from the VR VM:

Wed Aug  2 04:04:14 UTC 2017 Executing cloud-early-config
Wed Aug  2 04:04:14 UTC 2017 Detected that we are running inside vmware guest
Wed Aug  2 04:04:14 UTC 2017 Patching  scripts oldmd5=3bd71d0ff4892529f127a46cb76d9aee newmd5=bfdc7753040c29810cfb707d21cb1cc8
Wed Aug  2 04:04:14 UTC 2017 Patching  cloud service
Wed Aug  2 04:04:15 UTC 2017 Rebooting system since we patched init scripts
Wed Aug  2 04:04:17 UTC 2017 Updating log4j-cloud.xml
Wed Aug  2 04:04:17 UTC 2017 Setting up dhcp server system vm
Wed Aug  2 04:04:17 UTC 2017 checking that eth0 has IP
Wed Aug  2 04:04:17 UTC 2017 checking that eth1 has IP
Wed Aug  2 04:04:18 UTC 2017 waiting for eth1 interface setup with ip timer=0
Wed Aug  2 04:04:38 UTC 2017 Executing cloud-early-config
Wed Aug  2 04:04:38 UTC 2017 Detected that we are running inside vmware guest
Wed Aug  2 04:04:39 UTC 2017 Patching  cloud service
Wed Aug  2 04:04:39 UTC 2017 Updating log4j-cloud.xml
Wed Aug  2 04:04:39 UTC 2017 Setting up dhcp server system vm
Wed Aug  2 04:04:39 UTC 2017 checking that eth0 has IP
Wed Aug  2 04:04:40 UTC 2017 waiting for eth0 interface setup with ip timer=0
Wed Aug  2 04:04:40 UTC 2017 checking that eth1 has IP
Wed Aug  2 04:04:42 UTC 2017 waiting for eth1 interface setup with ip timer=0
Wed Aug  2 04:04:43 UTC 2017 waiting for eth1 interface setup with ip timer=1
Wed Aug  2 04:04:44 UTC 2017 waiting for eth1 interface setup with ip timer=2
Wed Aug  2 04:04:45 UTC 2017 waiting for eth1 interface setup with ip timer=3
Wed Aug  2 04:04:46 UTC 2017 waiting for eth1 interface setup with ip timer=4
Wed Aug  2 04:04:47 UTC 2017 waiting for eth1 interface setup with ip timer=5
Wed Aug  2 04:04:48 UTC 2017 waiting for eth1 interface setup with ip timer=6
Wed Aug  2 04:04:49 UTC 2017 waiting for eth1 interface setup with ip timer=7
Wed Aug  2 04:04:50 UTC 2017 waiting for eth1 interface setup with ip timer=8
Wed Aug  2 04:04:51 UTC 2017 waiting for eth1 interface setup with ip timer=9
Wed Aug  2 04:04:52 UTC 2017 waiting for eth1 interface setup with ip timer=10
Wed Aug  2 04:04:53 UTC 2017 waiting for eth1 interface setup with ip timer=11
Wed Aug  2 04:04:54 UTC 2017 waiting for eth1 interface setup with ip timer=12
Wed Aug  2 04:04:55 UTC 2017 waiting for eth1 interface setup with ip timer=13
Wed Aug  2 04:04:56 UTC 2017 waiting for eth1 interface setup with ip timer=14
Wed Aug  2 04:04:57 UTC 2017 waiting for eth1 interface setup with ip timer=15
Wed Aug  2 04:04:58 UTC 2017 waiting for eth1 interface setup with ip timer=16
Wed Aug  2 04:04:58 UTC 2017 interface eth1 is not set up with ip... exiting
Wed Aug  2 04:05:07 UTC 2017 Setting up dnsmasq
Wed Aug  2 04:05:07 UTC 2017 Setting up apache web server
Wed Aug  2 04:05:07 UTC 2017 Enable service dnsmasq = 1
Wed Aug  2 04:05:07 UTC 2017 Enable service haproxy = 0
Wed Aug  2 04:05:07 UTC 2017 Processors = 1  Enable service  = 0
Wed Aug  2 04:05:07 UTC 2017 Enable service cloud-passwd-srvr = 1
Wed Aug  2 04:05:07 UTC 2017 Enable service cloud = 0
Wed Aug  2 04:05:08 UTC 2017 cloud: enable_fwding = 0
Wed Aug  2 04:05:08 UTC 2017 enable_fwding = 0
2017-08-02 04:05:08,447  merge.py process:101 Command of type cmdline received
2017-08-02 04:05:08,447  merge.py process:101 Command of type ips received
2017-08-02 04:05:08,448  merge.py process:101 Command of type ips received
2017-08-02 04:05:08,451  merge.py load:291 Could not open /var/cache/cloud/cmd_line.json
2017-08-02 04:05:08,465  CsAddress.py process:107 Address found in DataBag ==> {u'public_ip':
u'0.0.0.0', u'one_to_one_nat': False, u'nic_dev_id': u'1', u'network': u'0.0.0.0/0', u'netmask':
u'0.0.0.0', u'source_nat': False, u'broadcast': u'255.255.255.255', u'add': True, u'nw_type':
u'control', u'device': u'eth1', u'cidr': u'0.0.0.0/0', u'gateway': u'10.117.40.126', u'size':
u'0'}
2017-08-02 04:05:08,465  CsAddress.py process:120 Address 0.0.0.0/0 on device eth1 not configured
2017-08-02 04:05:08,466  CsAddress.py configure:278 Configuring address 0.0.0.0/0 on device
eth1
2017-08-02 04:05:08,470  CsRoute.py defaultroute_exists:104 Checking if default ipv4 route
is present
2017-08-02 04:05:08,475  CsRoute.py defaultroute_exists:108 Default route found: default via
10.117.40.126 dev eth0
2017-08-02 04:05:08,479  CsAddress.py process:107 Address found in DataBag ==> {u'public_ip':
u'10.117.40.29', u'one_to_one_nat': False, u'nic_dev_id': u'0', u'network': u'10.117.40.0/25',
u'netmask': u'255.255.255.128', u'source_nat': False, u'broadcast': u'10.117.40.127', u'add':
True, u'nw_type': u'guest', u'device': u'eth0', u'cidr': u'10.117.40.29/25', u'gateway': u'10.117.40.126',
u'size': u'25'}
2017-08-02 04:05:08,480  CsAddress.py process:115 Address 10.117.40.29/25 on device eth0 already
configured
2017-08-02 04:05:08,480  CsRoute.py add_table:38 Adding route table: 0 Table_eth0 to /etc/iproute2/rt_tables
if not present
2017-08-02 04:05:08,548  CsRule.py addMark:49 Added fwmark rule for Table_eth0
2017-08-02 04:05:08,562  CsRoute.py add_route:55 Adding route: dev eth0 table: Table_eth0
network: 10.117.40.126 if not present
2017-08-02 04:05:08,568  CsRoute.py set_route:75 Add default via 10.117.40.126 table Table_eth0
proto static
2017-08-02 04:05:08,573  CsRoute.py add_network_route:65 Adding route: dev eth0 table: Table_eth0
network: 10.117.40.0/25 if not present
2017-08-02 04:05:08,589  CsHelper.py copy:254 Copied /etc/apache2/vhost.template to /etc/apache2/sites-enabled/vhost-10.117.40.29.conf
2017-08-02 04:05:08,592  CsFile.py commit:66 Wrote edited file /etc/apache2/sites-enabled/vhost-10.117.40.29.conf
2017-08-02 04:05:08,592  CsFile.py commit:68 Updated file in-cache configuration
2017-08-02 04:05:08,860  CsHelper.py service:216 Service apache2 restart
2017-08-02 04:05:08,876  CsProcess.py start:33 Started dummy
2017-08-02 04:05:08,878  CsRoute.py defaultroute_exists:104 Checking if default ipv4 route
is present
2017-08-02 04:05:08,885  CsRoute.py defaultroute_exists:108 Default route found: default via
10.117.40.126 dev eth0
2017-08-02 04:05:08,900  CsFile.py commit:66 Wrote edited file /etc/hosts
2017-08-02 04:05:08,900  CsFile.py commit:68 Updated file in-cache configuration
2017-08-02 04:05:08,900  CsDhcp.py write_hosts:128 Updated hosts file
2017-08-02 04:05:08,900  CsFile.py commit:60 Nothing to commit. The /etc/dnsmasq.d/cloud.conf
file did not change
2017-08-02 04:05:08,900  CsFile.py commit:60 Nothing to commit. The /etc/dhcphosts.txt file
did not change
2017-08-02 04:05:08,900  CsFile.py commit:60 Nothing to commit. The /etc/dhcpopts.txt file
did not change
2017-08-02 04:05:10,995  CsHelper.py service:216 Service dnsmasq restart
2017-08-02 04:05:11,156  CsNetfilter.py compare:163 Add: rule=-A FW_OUTBOUND -m state --state
RELATED,ESTABLISHED -j ACCEPT table=filter
2017-08-02 04:05:11,170  CsNetfilter.py compare:163 Add: rule=-A INPUT -i eth1 -p tcp -m tcp
--dport 3922 -m state --state NEW,ESTABLISHED -j ACCEPT table=filter
2017-08-02 04:05:11,183  CsNetfilter.py compare:163 Add: rule=-P INPUT DROP table=filter
2017-08-02 04:05:11,188  CsNetfilter.py compare:163 Add: rule=-P FORWARD DROP table=filter
2017-08-02 04:05:11,193  CsNetfilter.py compare:163 Add: rule=-A PREROUTING -m state --state
RELATED,ESTABLISHED -j CONNMARK --restore-mark --nfmask 0xffffffff --ctmask 0xffffffff table=mangle
2017-08-02 04:05:11,202  CsNetfilter.py compare:163 Add: rule=-A POSTROUTING -p udp -m udp
--dport 68 -j CHECKSUM --checksum-fill table=mangle
2017-08-02 04:05:11,213  CsNetfilter.py compare:163 Add: rule=-A INPUT -d 224.0.0.18/32 -j
ACCEPT table=filter
2017-08-02 04:05:11,218  CsNetfilter.py compare:163 Add: rule=-A INPUT -d 225.0.0.50/32 -j
ACCEPT table=filter
2017-08-02 04:05:11,223  CsNetfilter.py compare:163 Add: rule=-A INPUT -i eth0 -m state --state
RELATED,ESTABLISHED -j ACCEPT table=filter
2017-08-02 04:05:11,228  CsNetfilter.py compare:163 Add: rule=-A INPUT -p icmp -j ACCEPT table=filter
2017-08-02 04:05:11,233  CsNetfilter.py compare:163 Add: rule=-A INPUT -i lo -j ACCEPT table=filter
2017-08-02 04:05:11,238  CsNetfilter.py compare:163 Add: rule=-A INPUT -i eth0 -p udp -m udp
--dport 67 -j ACCEPT table=filter
2017-08-02 04:05:11,243  CsNetfilter.py compare:163 Add: rule=-A INPUT -i eth0 -p udp -m udp
--dport 53 -s 10.117.40.0/25 -j ACCEPT table=filter
2017-08-02 04:05:11,248  CsNetfilter.py compare:163 Add: rule=-A INPUT -i eth0 -p tcp -m tcp
--dport 53 -s 10.117.40.0/25 -j ACCEPT table=filter
2017-08-02 04:05:11,253  CsNetfilter.py compare:163 Add: rule=-A INPUT -i eth0 -p tcp -m tcp
--dport 80 -m state --state NEW -j ACCEPT table=filter
2017-08-02 04:05:11,259  CsNetfilter.py compare:163 Add: rule=-A INPUT -i eth0 -p tcp -m tcp
--dport 8080 -m state --state NEW -j ACCEPT table=filter
2017-08-02 04:05:11,264  CsNetfilter.py compare:163 Add: rule=-A FORWARD -i eth0 -o eth1 -m
state --state RELATED,ESTABLISHED -j ACCEPT table=filter
2017-08-02 04:05:11,270  CsNetfilter.py compare:163 Add: rule=-A FORWARD -i eth0 -o eth0 -m
state --state NEW -j ACCEPT table=filter
2017-08-02 04:05:11,275  CsNetfilter.py compare:163 Add: rule=-A FORWARD -i eth0 -o eth0 -m
state --state RELATED,ESTABLISHED -j ACCEPT table=filter
2017-08-02 04:05:11,280  CsNetfilter.py compare:163 Add: rule=-A FORWARD -j NETWORK_STATS
table=filter
2017-08-02 04:05:11,285  CsNetfilter.py compare:163 Add: rule=-A INPUT -j NETWORK_STATS table=filter
2017-08-02 04:05:11,290  CsNetfilter.py compare:163 Add: rule=-A OUTPUT -j NETWORK_STATS table=filter
2017-08-02 04:05:11,296  CsNetfilter.py compare:163 Add: rule=-A NETWORK_STATS -i eth0 -o
eth2 table=filter
2017-08-02 04:05:11,301  CsNetfilter.py compare:163 Add: rule=-A NETWORK_STATS -i eth2 -o
eth0 table=filter
2017-08-02 04:05:11,306  CsNetfilter.py compare:163 Add: rule=-A NETWORK_STATS -o eth2 ! -i
eth0 -p tcp table=filter
2017-08-02 04:05:11,311  CsNetfilter.py compare:163 Add: rule=-A NETWORK_STATS -i eth2 ! -o
eth0 -p tcp table=filter
2017-08-02 04:05:11,317  CsNetfilter.py compare:163 Add: rule=-A INPUT -i eth0 -p udp -m udp
--dport 67 -j ACCEPT table=filter
2017-08-02 04:05:11,322  CsNetfilter.py compare:163 Add: rule=-A INPUT -i eth0 -d 10.117.40.29/32
-p udp -m udp --dport 53 -j ACCEPT table=filter
2017-08-02 04:05:11,327  CsNetfilter.py compare:163 Add: rule=-A INPUT -i eth0 -d 10.117.40.29/32
-p tcp -m tcp --dport 53 -j ACCEPT table=filter
2017-08-02 04:05:11,332  CsNetfilter.py compare:163 Add: rule=-A INPUT -i eth0 -d 10.117.40.29/32
-p tcp -m tcp -m state --state NEW --dport 80 -j ACCEPT table=filter
2017-08-02 04:05:11,338  CsNetfilter.py compare:163 Add: rule=-A INPUT -i eth0 -d 10.117.40.29/32
-p tcp -m tcp -m state --state NEW --dport 443 -j ACCEPT table=filter
2017-08-02 04:05:11,396  CsHelper.py service:216 Service conntrackd stop
2017-08-02 04:05:11,411  CsHelper.py service:216 Service keepalived stop

Thanks!
Mike
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message