cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "DeepthiMachiraju (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CLOUDSTACK-9959) [UI] Inconsistency in Redundant state switching over to Master when current master is destroyed
Date Tue, 13 Jun 2017 11:51:00 GMT

     [ https://issues.apache.org/jira/browse/CLOUDSTACK-9959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

DeepthiMachiraju updated CLOUDSTACK-9959:
-----------------------------------------
    Description: 
- In AD , create an Isolated NW with RVR Network offering.
- Once both the master and back up are in Running state , destroy the Master.

Observation :

- Back up is not switched to Master in the UI and DB .
- Whereas in VR the state shows as  Master..
- no CheckRouterCommand / CheckRouterAnswer observed for the new failover router in Ms log.
================================
root@r-135-VM:/etc/cloudstack# cat cmdline.json
{
    "config": {
        "--": "--",
        "advert_int": "1",
        "baremetalnotificationapikey": "xucbW6QYh6CBtoAWfm1IBNyIs62G3Zj0uhxA5mlxTP1fecZdcgl3PYjhP45fIBuc-JarB8r7H1obENvOQlF2iA",
        "baremetalnotificationsecuritykey": "xtLBtehx4T-XdhDfSaeRvvEJDwKAtYsjz9DwtstYrKBekJcQg1jPM7ThIR8BBpZr1nuYZVxoxrwkghNEE_xX5A",
        "cidrsize": "24",
        "console": "hvc0",
        "debian-installer": "en_US",
        "dhcprange": "10.1.1.1",
        "disable_rp_filter": "true",
        "dns1": "10.105.112.2",
        "domain": "cs2cloud.internal",
        "eth0ip": "10.1.1.146",
        "eth0mask": "255.255.255.0",
        "eth1ip": "169.254.2.200",
        "eth1mask": "255.255.0.0",
        "eth2ip": "10.147.30.115",
        "eth2mask": "255.255.255.0",
        "gateway": "10.147.30.1",
        "guestbrd": "10.1.1.255",
        "guestcidrsize": "24",
        "guestgw": "10.1.1.1",
        "host": "10.147.59.149",
        "name": "r-135-VM",
        "port": "8080",
        "quiet": "quiet",
        "redundant_master": true,
        "redundant_router": "true",
        "redundant_state": "MASTER",
        "ro": "ro",
        "root": "UUID",
        "template": "domP",
        "type": "router"
    },
    "id": "cmdline"
}

=============================
root@r-135-VM:/etc/cloudstack#
root@r-135-VM:/etc/cloudstack# checkrouter.sh
Status: MASTER
================================
mysql> select * from domain_router where id=135\G;
******* 1. row *********
                 id: 135
         element_id: 1
 public_mac_address: 06:58:d0:00:00:10
  public_ip_address: 10.147.30.115
     public_netmask: 255.255.255.0
      guest_netmask: NULL
   guest_ip_address: NULL
is_redundant_router: 1
           priority: NULL
    redundant_state: BACKUP
       stop_pending: 0
               role: VIRTUAL_ROUTER
   template_version: Cloudstack Release 4.10.0 Wed May 17 11:36:48 UTC 2017
    scripts_version: 296ed37908709aa4e1e9e04b03bb6248

             vpc_id: NULL
       update_state: NULL
1 row in set (0.00 sec)

===============cloud.log=================
[INFO] update_config.py :: Processing incoming file => vm_metadata.json
[INFO] Processing JSON file vm_metadata.json
Tue Jun 13 10:06:27 UTC 2017 : VR config: execution success
Tue Jun 13 10:06:27 UTC 2017 : VR config: Flushing conntrack table
Tue Jun 13 10:06:27 UTC 2017 : VR config: Flushing conntrack table completed
2017-06-13 10:08:38,045 DEBUG    Loading data bag type cmdline
2017-06-13 10:08:38,046 DEBUG    Loading data bag type cmdline
2017-06-13 10:08:38,046 DEBUG    Writing data bag type cmdline
2017-06-13 10:08:38,046 DEBUG    {u'config': {u'domain': u'cs2cloud.internal', u'guestcidrsize':
u'24', u'cidrsize': u'24', u'eth1ip': u'169.254.2.200', u'redundant_master': False, u'gateway':
u'10.147.30.1', u'eth0ip': u'10.1.1.146', u'advert_int': u'1', u'console': u'hvc0', u'guestgw':
u'10.1.1.1', u'debian-installer': u'en_US', u'port': u'8080', u'guestbrd': u'10.1.1.255',
u'template': u'domP', u'ro': u'ro', u'type': u'router', u'eth2mask': u'255.255.255.0', u'baremetalnotificationsecuritykey':
u'xtLBtehx4T-XdhDfSaeRvvEJDwKAtYsjz9DwtstYrKBekJcQg1jPM7ThIR8BBpZr1nuYZVxoxrwkghNEE_xX5A',
u'eth0mask': u'255.255.255.0', u'eth1mask': u'255.255.0.0', u'dhcprange': u'10.1.1.1', u'eth2ip':
u'10.147.30.115', u'host': u'10.147.59.149', u'redundant_router': u'true', u'baremetalnotificationapikey':
u'xucbW6QYh6CBtoAWfm1IBNyIs62G3Zj0uhxA5mlxTP1fecZdcgl3PYjhP45fIBuc-JarB8r7H1obENvOQlF2iA',
u'redundant_state': 'BACKUP', u'name': u'r-135-VM', u'--': u'--', u'dns1': u'10.105.112.2',
u'quiet': u'quiet', u'disable_rp_filter': u'true', u'root': u'UUID'}, u'id': u'cmdline'}
2017-06-13 10:08:38,046 DEBUG    Loading data bag type ips
2017-06-13 10:08:38,046 DEBUG    Setting router to master
2017-06-13 10:08:38,046 INFO     Will proceed configuring device ==> eth2
2017-06-13 10:08:38,047 DEBUG    Executing: ip link set eth2 up
2017-06-13 10:08:38,052 INFO     Bringing public interface eth2 up
2017-06-13 10:08:38,053 INFO     Adding gateway ==> 10.147.30.1 to device ==> eth2
2017-06-13 10:08:38,053 INFO     Checking if default ipv4 route is present
2017-06-13 10:08:38,053 DEBUG    Executing: ip -4 route list 0/0
2017-06-13 10:08:38,057 WARNING  No default route found!
2017-06-13 10:08:38,058 INFO     Adding default route
2017-06-13 10:08:38,058 DEBUG    Executing: ip route show default via 10.147.30.1
2017-06-13 10:08:38,062 INFO     Add default via 10.147.30.1
2017-06-13 10:08:38,063 DEBUG    Executing: ip route add default via 10.147.30.1
2017-06-13 10:08:38,067 DEBUG    Configuring static routes
2017-06-13 10:08:38,068 DEBUG    Creating data bag type staticroutes
2017-06-13 10:08:38,068 DEBUG    Processing CsStaticRoutes file ==> {'id': 'staticroutes'}
2017-06-13 10:08:38,068 DEBUG    Executing: /usr/sbin/conntrackd -C /etc/conntrackd/conntrackd.conf
-c
2017-06-13 10:08:38,073 DEBUG    Executing: /usr/sbin/conntrackd -C /etc/conntrackd/conntrackd.conf
-f
2017-06-13 10:08:38,079 DEBUG    Executing: /usr/sbin/conntrackd -C /etc/conntrackd/conntrackd.conf
-R
2017-06-13 10:08:38,084 DEBUG    Executing: /usr/sbin/conntrackd -C /etc/conntrackd/conntrackd.conf
-B
2017-06-13 10:08:38,089 DEBUG    Executing: service ipsec restart
2017-06-13 10:08:40,413 INFO     Service ipsec restart
2017-06-13 10:08:40,413 DEBUG    Executing: service xl2tpd restart
2017-06-13 10:08:42,012 INFO     Service xl2tpd restart
2017-06-13 10:08:42,063 DEBUG    Executing: ps aux
2017-06-13 10:08:42,073 DEBUG    Executing: ps aux
2017-06-13 10:08:42,082 DEBUG    Executing: ps aux
2017-06-13 10:08:42,092 INFO     Started dummy
2017-06-13 10:08:42,095 DEBUG    Executing: service dnsmasq restart
2017-06-13 10:08:44,118 INFO     Service dnsmasq restart
2017-06-13 10:08:44,118 DEBUG    Writing data bag type cmdline
2017-06-13 10:08:44,118 DEBUG    {u'config': {u'domain': u'cs2cloud.internal', u'guestcidrsize':
u'24', u'cidrsize': u'24', u'eth1ip': u'169.254.2.200', u'redundant_master': True, u'gateway':
u'10.147.30.1', u'eth0ip': u'10.1.1.146', u'advert_int': u'1', u'console': u'hvc0', u'guestgw':
u'10.1.1.1', u'debian-installer': u'en_US', u'port': u'8080', u'guestbrd': u'10.1.1.255',
u'template': u'domP', u'ro': u'ro', u'type': u'router', u'eth2mask': u'255.255.255.0', u'baremetalnotificationsecuritykey':
u'xtLBtehx4T-XdhDfSaeRvvEJDwKAtYsjz9DwtstYrKBekJcQg1jPM7ThIR8BBpZr1nuYZVxoxrwkghNEE_xX5A',
u'eth0mask': u'255.255.255.0', u'eth1mask': u'255.255.0.0', u'dhcprange': u'10.1.1.1', u'eth2ip':
u'10.147.30.115', u'host': u'10.147.59.149', u'redundant_router': u'true', u'baremetalnotificationapikey':
u'xucbW6QYh6CBtoAWfm1IBNyIs62G3Zj0uhxA5mlxTP1fecZdcgl3PYjhP45fIBuc-JarB8r7H1obENvOQlF2iA',
u'redundant_state': 'MASTER', u'name': u'r-135-VM', u'--': u'--', u'dns1': u'10.105.112.2',
u'quiet': u'quiet', u'disable_rp_filter': u'true', u'root': u'UUID'}, u'id': u'cmdline'}
2017-06-13 10:08:44,118 DEBUG    Executing: ip link show eth2 | grep 'state DOWN'
2017-06-13 10:08:44,125 INFO     Router switched to master mode




  was:
- In AD , create an Isolated NW with RVR Network offering.
- Once both the master and back up are in Running state , destroy the Master.

Observation :

- Back up is not switched to Master in the UI and DB .
- Whereas in VR the state shows as  Master..
- no CheckRouterCommand / CheckRouterAnswer observed for the new failover router in Ms log.
================================
root@r-135-VM:/etc/cloudstack# cat cmdline.json
{
    "config": {
        "--": "--",
        "advert_int": "1",
        "baremetalnotificationapikey": "xucbW6QYh6CBtoAWfm1IBNyIs62G3Zj0uhxA5mlxTP1fecZdcgl3PYjhP45fIBuc-JarB8r7H1obENvOQlF2iA",
        "baremetalnotificationsecuritykey": "xtLBtehx4T-XdhDfSaeRvvEJDwKAtYsjz9DwtstYrKBekJcQg1jPM7ThIR8BBpZr1nuYZVxoxrwkghNEE_xX5A",
        "cidrsize": "24",
        "console": "hvc0",
        "debian-installer": "en_US",
        "dhcprange": "10.1.1.1",
        "disable_rp_filter": "true",
        "dns1": "10.105.112.2",
        "domain": "cs2cloud.internal",
        "eth0ip": "10.1.1.146",
        "eth0mask": "255.255.255.0",
        "eth1ip": "169.254.2.200",
        "eth1mask": "255.255.0.0",
        "eth2ip": "10.147.30.115",
        "eth2mask": "255.255.255.0",
        "gateway": "10.147.30.1",
        "guestbrd": "10.1.1.255",
        "guestcidrsize": "24",
        "guestgw": "10.1.1.1",
        "host": "10.147.59.149",
        "name": "r-135-VM",
        "port": "8080",
        "quiet": "quiet",
        "redundant_master": true,
        "redundant_router": "true",
        "redundant_state": "MASTER",
        "ro": "ro",
        "root": "UUID",
        "template": "domP",
        "type": "router"
    },
    "id": "cmdline"
}

=============================
root@r-135-VM:/etc/cloudstack#
root@r-135-VM:/etc/cloudstack# checkrouter.sh
Status: MASTER
================================
mysql> select * from domain_router where id=135\G;
******* 1. row *********
                 id: 135
         element_id: 1
 public_mac_address: 06:58:d0:00:00:10
  public_ip_address: 10.147.30.115
     public_netmask: 255.255.255.0
      guest_netmask: NULL
   guest_ip_address: NULL
is_redundant_router: 1
           priority: NULL
    redundant_state: BACKUP
       stop_pending: 0
               role: VIRTUAL_ROUTER
   template_version: Cloudstack Release 4.10.0 Wed May 17 11:36:48 UTC 2017
    scripts_version: 296ed37908709aa4e1e9e04b03bb6248

             vpc_id: NULL
       update_state: NULL
1 row in set (0.00 sec)




> [UI] Inconsistency in Redundant state switching over to Master when current master is
destroyed
> -----------------------------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-9959
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9959
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: Virtual Router
>    Affects Versions: 4.10.0.0
>            Reporter: DeepthiMachiraju
>              Labels: PVR
>             Fix For: 4.10.0.0
>
>
> - In AD , create an Isolated NW with RVR Network offering.
> - Once both the master and back up are in Running state , destroy the Master.
> Observation :
> - Back up is not switched to Master in the UI and DB .
> - Whereas in VR the state shows as  Master..
> - no CheckRouterCommand / CheckRouterAnswer observed for the new failover router in Ms
log.
> ================================
> root@r-135-VM:/etc/cloudstack# cat cmdline.json
> {
>     "config": {
>         "--": "--",
>         "advert_int": "1",
>         "baremetalnotificationapikey": "xucbW6QYh6CBtoAWfm1IBNyIs62G3Zj0uhxA5mlxTP1fecZdcgl3PYjhP45fIBuc-JarB8r7H1obENvOQlF2iA",
>         "baremetalnotificationsecuritykey": "xtLBtehx4T-XdhDfSaeRvvEJDwKAtYsjz9DwtstYrKBekJcQg1jPM7ThIR8BBpZr1nuYZVxoxrwkghNEE_xX5A",
>         "cidrsize": "24",
>         "console": "hvc0",
>         "debian-installer": "en_US",
>         "dhcprange": "10.1.1.1",
>         "disable_rp_filter": "true",
>         "dns1": "10.105.112.2",
>         "domain": "cs2cloud.internal",
>         "eth0ip": "10.1.1.146",
>         "eth0mask": "255.255.255.0",
>         "eth1ip": "169.254.2.200",
>         "eth1mask": "255.255.0.0",
>         "eth2ip": "10.147.30.115",
>         "eth2mask": "255.255.255.0",
>         "gateway": "10.147.30.1",
>         "guestbrd": "10.1.1.255",
>         "guestcidrsize": "24",
>         "guestgw": "10.1.1.1",
>         "host": "10.147.59.149",
>         "name": "r-135-VM",
>         "port": "8080",
>         "quiet": "quiet",
>         "redundant_master": true,
>         "redundant_router": "true",
>         "redundant_state": "MASTER",
>         "ro": "ro",
>         "root": "UUID",
>         "template": "domP",
>         "type": "router"
>     },
>     "id": "cmdline"
> }
> =============================
> root@r-135-VM:/etc/cloudstack#
> root@r-135-VM:/etc/cloudstack# checkrouter.sh
> Status: MASTER
> ================================
> mysql> select * from domain_router where id=135\G;
> ******* 1. row *********
>                  id: 135
>          element_id: 1
>  public_mac_address: 06:58:d0:00:00:10
>   public_ip_address: 10.147.30.115
>      public_netmask: 255.255.255.0
>       guest_netmask: NULL
>    guest_ip_address: NULL
> is_redundant_router: 1
>            priority: NULL
>     redundant_state: BACKUP
>        stop_pending: 0
>                role: VIRTUAL_ROUTER
>    template_version: Cloudstack Release 4.10.0 Wed May 17 11:36:48 UTC 2017
>     scripts_version: 296ed37908709aa4e1e9e04b03bb6248
>              vpc_id: NULL
>        update_state: NULL
> 1 row in set (0.00 sec)
> ===============cloud.log=================
> [INFO] update_config.py :: Processing incoming file => vm_metadata.json
> [INFO] Processing JSON file vm_metadata.json
> Tue Jun 13 10:06:27 UTC 2017 : VR config: execution success
> Tue Jun 13 10:06:27 UTC 2017 : VR config: Flushing conntrack table
> Tue Jun 13 10:06:27 UTC 2017 : VR config: Flushing conntrack table completed
> 2017-06-13 10:08:38,045 DEBUG    Loading data bag type cmdline
> 2017-06-13 10:08:38,046 DEBUG    Loading data bag type cmdline
> 2017-06-13 10:08:38,046 DEBUG    Writing data bag type cmdline
> 2017-06-13 10:08:38,046 DEBUG    {u'config': {u'domain': u'cs2cloud.internal', u'guestcidrsize':
u'24', u'cidrsize': u'24', u'eth1ip': u'169.254.2.200', u'redundant_master': False, u'gateway':
u'10.147.30.1', u'eth0ip': u'10.1.1.146', u'advert_int': u'1', u'console': u'hvc0', u'guestgw':
u'10.1.1.1', u'debian-installer': u'en_US', u'port': u'8080', u'guestbrd': u'10.1.1.255',
u'template': u'domP', u'ro': u'ro', u'type': u'router', u'eth2mask': u'255.255.255.0', u'baremetalnotificationsecuritykey':
u'xtLBtehx4T-XdhDfSaeRvvEJDwKAtYsjz9DwtstYrKBekJcQg1jPM7ThIR8BBpZr1nuYZVxoxrwkghNEE_xX5A',
u'eth0mask': u'255.255.255.0', u'eth1mask': u'255.255.0.0', u'dhcprange': u'10.1.1.1', u'eth2ip':
u'10.147.30.115', u'host': u'10.147.59.149', u'redundant_router': u'true', u'baremetalnotificationapikey':
u'xucbW6QYh6CBtoAWfm1IBNyIs62G3Zj0uhxA5mlxTP1fecZdcgl3PYjhP45fIBuc-JarB8r7H1obENvOQlF2iA',
u'redundant_state': 'BACKUP', u'name': u'r-135-VM', u'--': u'--', u'dns1': u'10.105.112.2',
u'quiet': u'quiet', u'disable_rp_filter': u'true', u'root': u'UUID'}, u'id': u'cmdline'}
> 2017-06-13 10:08:38,046 DEBUG    Loading data bag type ips
> 2017-06-13 10:08:38,046 DEBUG    Setting router to master
> 2017-06-13 10:08:38,046 INFO     Will proceed configuring device ==> eth2
> 2017-06-13 10:08:38,047 DEBUG    Executing: ip link set eth2 up
> 2017-06-13 10:08:38,052 INFO     Bringing public interface eth2 up
> 2017-06-13 10:08:38,053 INFO     Adding gateway ==> 10.147.30.1 to device ==> eth2
> 2017-06-13 10:08:38,053 INFO     Checking if default ipv4 route is present
> 2017-06-13 10:08:38,053 DEBUG    Executing: ip -4 route list 0/0
> 2017-06-13 10:08:38,057 WARNING  No default route found!
> 2017-06-13 10:08:38,058 INFO     Adding default route
> 2017-06-13 10:08:38,058 DEBUG    Executing: ip route show default via 10.147.30.1
> 2017-06-13 10:08:38,062 INFO     Add default via 10.147.30.1
> 2017-06-13 10:08:38,063 DEBUG    Executing: ip route add default via 10.147.30.1
> 2017-06-13 10:08:38,067 DEBUG    Configuring static routes
> 2017-06-13 10:08:38,068 DEBUG    Creating data bag type staticroutes
> 2017-06-13 10:08:38,068 DEBUG    Processing CsStaticRoutes file ==> {'id': 'staticroutes'}
> 2017-06-13 10:08:38,068 DEBUG    Executing: /usr/sbin/conntrackd -C /etc/conntrackd/conntrackd.conf
-c
> 2017-06-13 10:08:38,073 DEBUG    Executing: /usr/sbin/conntrackd -C /etc/conntrackd/conntrackd.conf
-f
> 2017-06-13 10:08:38,079 DEBUG    Executing: /usr/sbin/conntrackd -C /etc/conntrackd/conntrackd.conf
-R
> 2017-06-13 10:08:38,084 DEBUG    Executing: /usr/sbin/conntrackd -C /etc/conntrackd/conntrackd.conf
-B
> 2017-06-13 10:08:38,089 DEBUG    Executing: service ipsec restart
> 2017-06-13 10:08:40,413 INFO     Service ipsec restart
> 2017-06-13 10:08:40,413 DEBUG    Executing: service xl2tpd restart
> 2017-06-13 10:08:42,012 INFO     Service xl2tpd restart
> 2017-06-13 10:08:42,063 DEBUG    Executing: ps aux
> 2017-06-13 10:08:42,073 DEBUG    Executing: ps aux
> 2017-06-13 10:08:42,082 DEBUG    Executing: ps aux
> 2017-06-13 10:08:42,092 INFO     Started dummy
> 2017-06-13 10:08:42,095 DEBUG    Executing: service dnsmasq restart
> 2017-06-13 10:08:44,118 INFO     Service dnsmasq restart
> 2017-06-13 10:08:44,118 DEBUG    Writing data bag type cmdline
> 2017-06-13 10:08:44,118 DEBUG    {u'config': {u'domain': u'cs2cloud.internal', u'guestcidrsize':
u'24', u'cidrsize': u'24', u'eth1ip': u'169.254.2.200', u'redundant_master': True, u'gateway':
u'10.147.30.1', u'eth0ip': u'10.1.1.146', u'advert_int': u'1', u'console': u'hvc0', u'guestgw':
u'10.1.1.1', u'debian-installer': u'en_US', u'port': u'8080', u'guestbrd': u'10.1.1.255',
u'template': u'domP', u'ro': u'ro', u'type': u'router', u'eth2mask': u'255.255.255.0', u'baremetalnotificationsecuritykey':
u'xtLBtehx4T-XdhDfSaeRvvEJDwKAtYsjz9DwtstYrKBekJcQg1jPM7ThIR8BBpZr1nuYZVxoxrwkghNEE_xX5A',
u'eth0mask': u'255.255.255.0', u'eth1mask': u'255.255.0.0', u'dhcprange': u'10.1.1.1', u'eth2ip':
u'10.147.30.115', u'host': u'10.147.59.149', u'redundant_router': u'true', u'baremetalnotificationapikey':
u'xucbW6QYh6CBtoAWfm1IBNyIs62G3Zj0uhxA5mlxTP1fecZdcgl3PYjhP45fIBuc-JarB8r7H1obENvOQlF2iA',
u'redundant_state': 'MASTER', u'name': u'r-135-VM', u'--': u'--', u'dns1': u'10.105.112.2',
u'quiet': u'quiet', u'disable_rp_filter': u'true', u'root': u'UUID'}, u'id': u'cmdline'}
> 2017-06-13 10:08:44,118 DEBUG    Executing: ip link show eth2 | grep 'state DOWN'
> 2017-06-13 10:08:44,125 INFO     Router switched to master mode



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message