cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wilder Rodrigues (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CLOUDSTACK-9154) rVPC doesn't recover from cleaning up of network garbage collector
Date Wed, 16 Dec 2015 14:33:46 GMT

     [ https://issues.apache.org/jira/browse/CLOUDSTACK-9154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Wilder Rodrigues updated CLOUDSTACK-9154:
-----------------------------------------
    Description: 
- deploy a rVPC
- deploy VM in it
- make port forwarding (2nd ip, firewall and such)
- confirm it works
- stop the vm
- after some time the network garbage collector will come and tear down the network since
there are no more VMs
- keepalived will enter FAULT state because of missing eth2 nic (which was first network tier)
- all is left is ethic (link local) and lo0
- then start the vm again
- the nics get plugged again and keepalived will decide on a new master
- the nics are screwed up after this:

```
root@r-1021-VM:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 0e:00:a9:fe:02:92 brd ff:ff:ff:ff:ff:ff
    inet 169.254.2.146/16 brd 169.254.255.255 scope global eth0
5: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 02:00:18:34:00:05 brd ff:ff:ff:ff:ff:ff
    inet x.y.238.24/24 brd x.y.238.255 scope global eth1
    inet 10.0.0.51/24 brd 10.0.0.255 scope global eth1
    inet 10.0.0.1/24 brd 10.0.0.255 scope global secondary eth1
6: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 06:d5:4e:00:00:1d brd ff:ff:ff:ff:ff:ff
    inet x.y.238.25/24 brd x.y.238.255 scope global eth2
    inet 10.0.0.1/24 brd 10.0.0.255 scope global eth2
root@r-1021-VM:~#
```

Public and tier ip addresses are mixed up.

/etc/cloudstack/ips.json has the wrong info:
```
{                                                                                        
                                                                                         
             [44/959]
    "eth0": [
        {
            "add": true,
            "broadcast": "169.254.255.255",
            "cidr": "169.254.2.146/16",
            "device": "eth0",
            "gateway": "None",
            "netmask": "255.255.0.0",
            "network": "169.254.0.0/16",
            "nic_dev_id": "0",
            "nw_type": "control",
            "one_to_one_nat": false,
            "public_ip": "169.254.2.146",
            "size": "16",
            "source_nat": false
        }
    ],
    "eth1": [
        {
            "add": true,
            "broadcast": "x.y.238.255",
            "cidr": "x.y.238.24/24",
            "device": "eth1",
            "first_i_p": true,
            "gateway": "x.y.238.1",
            "netmask": "255.255.255.0",
            "network": "x.y.238.0/24",
            "new_nic": false,
            "nic_dev_id": 1,
            "nw_type": "public",
            "one_to_one_nat": false,
            "public_ip": "x.y.238.24",
            "size": "24",
            "source_nat": true,
            "vif_mac_address": "06:fc:da:00:00:1c"
        },
        {
            "add": true,
            "broadcast": "10.0.0.255",
            "cidr": "10.0.0.51/24",
            "device": "eth1",
            "gateway": "10.0.0.1",
            "netmask": "255.255.255.0",
            "network": "10.0.0.0/24",
            "nic_dev_id": "1",
            "nw_type": "guest",
            "one_to_one_nat": false,
            "public_ip": "10.0.0.51",
            "size": "24",
            "source_nat": false
        }
    ],
    "eth2": [
        {
            "add": false,
            "broadcast": "10.0.0.255",
            "cidr": "10.0.0.173/24",
            "device": "eth2",
            "gateway": "10.0.0.1",
            "netmask": "255.255.255.0",
            "network": "10.0.0.0/24",
            "nic_dev_id": "2",
            "nw_type": "guest",
            "one_to_one_nat": false,
            "public_ip": "10.0.0.173",
            "size": "24",
            "source_nat": false
        },
        {
            "add": true,
            "broadcast": "x.y.238.255",
            "cidr": "x.y.238.25/24",
            "device": "eth2",
            "first_i_p": true,
            "gateway": "x.y.238.1",
            "netmask": "255.255.255.0",
            "network": "x.y.238.0/24",
            "new_nic": false,
            "nic_dev_id": 2,
            "nw_type": "public",
            "one_to_one_nat": false,
            "public_ip": "x.y.238.25",
            "size": "24",
            "source_nat": true,
            "vif_mac_address": "06:d5:4e:00:00:1d"
        }
    ],
    "id": "ips"
```

Pinging [~wilder.rodrigues]

  was:
- deploy a rVPC
- deploy VM in it
- make port forwarding (2nd ip, firewall and such)
- confirm it works
- stop the vm
- after some time the network scavenger will come and tear down the network since there are
no more VMs
- keepalived will enter FAULT state because of missing eth2 nic (which was first network tier)
- all is left is ethic (link local) and lo0
- then start the vm again
- the nics get plugged again and keepalived will decide on a new master
- the nics are screwed up after this:

```
root@r-1021-VM:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 0e:00:a9:fe:02:92 brd ff:ff:ff:ff:ff:ff
    inet 169.254.2.146/16 brd 169.254.255.255 scope global eth0
5: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 02:00:18:34:00:05 brd ff:ff:ff:ff:ff:ff
    inet x.y.238.24/24 brd x.y.238.255 scope global eth1
    inet 10.0.0.51/24 brd 10.0.0.255 scope global eth1
    inet 10.0.0.1/24 brd 10.0.0.255 scope global secondary eth1
6: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 06:d5:4e:00:00:1d brd ff:ff:ff:ff:ff:ff
    inet x.y.238.25/24 brd x.y.238.255 scope global eth2
    inet 10.0.0.1/24 brd 10.0.0.255 scope global eth2
root@r-1021-VM:~#
```

Public and tier ip addresses are mixed up.

/etc/cloudstack/ips.json has the wrong info:
```
{                                                                                        
                                                                                         
             [44/959]
    "eth0": [
        {
            "add": true,
            "broadcast": "169.254.255.255",
            "cidr": "169.254.2.146/16",
            "device": "eth0",
            "gateway": "None",
            "netmask": "255.255.0.0",
            "network": "169.254.0.0/16",
            "nic_dev_id": "0",
            "nw_type": "control",
            "one_to_one_nat": false,
            "public_ip": "169.254.2.146",
            "size": "16",
            "source_nat": false
        }
    ],
    "eth1": [
        {
            "add": true,
            "broadcast": "x.y.238.255",
            "cidr": "x.y.238.24/24",
            "device": "eth1",
            "first_i_p": true,
            "gateway": "x.y.238.1",
            "netmask": "255.255.255.0",
            "network": "x.y.238.0/24",
            "new_nic": false,
            "nic_dev_id": 1,
            "nw_type": "public",
            "one_to_one_nat": false,
            "public_ip": "x.y.238.24",
            "size": "24",
            "source_nat": true,
            "vif_mac_address": "06:fc:da:00:00:1c"
        },
        {
            "add": true,
            "broadcast": "10.0.0.255",
            "cidr": "10.0.0.51/24",
            "device": "eth1",
            "gateway": "10.0.0.1",
            "netmask": "255.255.255.0",
            "network": "10.0.0.0/24",
            "nic_dev_id": "1",
            "nw_type": "guest",
            "one_to_one_nat": false,
            "public_ip": "10.0.0.51",
            "size": "24",
            "source_nat": false
        }
    ],
    "eth2": [
        {
            "add": false,
            "broadcast": "10.0.0.255",
            "cidr": "10.0.0.173/24",
            "device": "eth2",
            "gateway": "10.0.0.1",
            "netmask": "255.255.255.0",
            "network": "10.0.0.0/24",
            "nic_dev_id": "2",
            "nw_type": "guest",
            "one_to_one_nat": false,
            "public_ip": "10.0.0.173",
            "size": "24",
            "source_nat": false
        },
        {
            "add": true,
            "broadcast": "x.y.238.255",
            "cidr": "x.y.238.25/24",
            "device": "eth2",
            "first_i_p": true,
            "gateway": "x.y.238.1",
            "netmask": "255.255.255.0",
            "network": "x.y.238.0/24",
            "new_nic": false,
            "nic_dev_id": 2,
            "nw_type": "public",
            "one_to_one_nat": false,
            "public_ip": "x.y.238.25",
            "size": "24",
            "source_nat": true,
            "vif_mac_address": "06:d5:4e:00:00:1d"
        }
    ],
    "id": "ips"
```

Pinging [~wilder.rodrigues]


> rVPC doesn't recover from cleaning up of network garbage collector
> ------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-9154
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9154
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: Virtual Router
>         Environment: ACS 4.7
>            Reporter: Remi Bergsma
>            Assignee: Wilder Rodrigues
>
> - deploy a rVPC
> - deploy VM in it
> - make port forwarding (2nd ip, firewall and such)
> - confirm it works
> - stop the vm
> - after some time the network garbage collector will come and tear down the network since
there are no more VMs
> - keepalived will enter FAULT state because of missing eth2 nic (which was first network
tier)
> - all is left is ethic (link local) and lo0
> - then start the vm again
> - the nics get plugged again and keepalived will decide on a new master
> - the nics are screwed up after this:
> ```
> root@r-1021-VM:~# ip a
> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
>     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
>     inet 127.0.0.1/8 scope host lo
> 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen
1000
>     link/ether 0e:00:a9:fe:02:92 brd ff:ff:ff:ff:ff:ff
>     inet 169.254.2.146/16 brd 169.254.255.255 scope global eth0
> 5: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen
1000
>     link/ether 02:00:18:34:00:05 brd ff:ff:ff:ff:ff:ff
>     inet x.y.238.24/24 brd x.y.238.255 scope global eth1
>     inet 10.0.0.51/24 brd 10.0.0.255 scope global eth1
>     inet 10.0.0.1/24 brd 10.0.0.255 scope global secondary eth1
> 6: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen
1000
>     link/ether 06:d5:4e:00:00:1d brd ff:ff:ff:ff:ff:ff
>     inet x.y.238.25/24 brd x.y.238.255 scope global eth2
>     inet 10.0.0.1/24 brd 10.0.0.255 scope global eth2
> root@r-1021-VM:~#
> ```
> Public and tier ip addresses are mixed up.
> /etc/cloudstack/ips.json has the wrong info:
> ```
> {                                                                                   
                                                                                         
                  [44/959]
>     "eth0": [
>         {
>             "add": true,
>             "broadcast": "169.254.255.255",
>             "cidr": "169.254.2.146/16",
>             "device": "eth0",
>             "gateway": "None",
>             "netmask": "255.255.0.0",
>             "network": "169.254.0.0/16",
>             "nic_dev_id": "0",
>             "nw_type": "control",
>             "one_to_one_nat": false,
>             "public_ip": "169.254.2.146",
>             "size": "16",
>             "source_nat": false
>         }
>     ],
>     "eth1": [
>         {
>             "add": true,
>             "broadcast": "x.y.238.255",
>             "cidr": "x.y.238.24/24",
>             "device": "eth1",
>             "first_i_p": true,
>             "gateway": "x.y.238.1",
>             "netmask": "255.255.255.0",
>             "network": "x.y.238.0/24",
>             "new_nic": false,
>             "nic_dev_id": 1,
>             "nw_type": "public",
>             "one_to_one_nat": false,
>             "public_ip": "x.y.238.24",
>             "size": "24",
>             "source_nat": true,
>             "vif_mac_address": "06:fc:da:00:00:1c"
>         },
>         {
>             "add": true,
>             "broadcast": "10.0.0.255",
>             "cidr": "10.0.0.51/24",
>             "device": "eth1",
>             "gateway": "10.0.0.1",
>             "netmask": "255.255.255.0",
>             "network": "10.0.0.0/24",
>             "nic_dev_id": "1",
>             "nw_type": "guest",
>             "one_to_one_nat": false,
>             "public_ip": "10.0.0.51",
>             "size": "24",
>             "source_nat": false
>         }
>     ],
>     "eth2": [
>         {
>             "add": false,
>             "broadcast": "10.0.0.255",
>             "cidr": "10.0.0.173/24",
>             "device": "eth2",
>             "gateway": "10.0.0.1",
>             "netmask": "255.255.255.0",
>             "network": "10.0.0.0/24",
>             "nic_dev_id": "2",
>             "nw_type": "guest",
>             "one_to_one_nat": false,
>             "public_ip": "10.0.0.173",
>             "size": "24",
>             "source_nat": false
>         },
>         {
>             "add": true,
>             "broadcast": "x.y.238.255",
>             "cidr": "x.y.238.25/24",
>             "device": "eth2",
>             "first_i_p": true,
>             "gateway": "x.y.238.1",
>             "netmask": "255.255.255.0",
>             "network": "x.y.238.0/24",
>             "new_nic": false,
>             "nic_dev_id": 2,
>             "nw_type": "public",
>             "one_to_one_nat": false,
>             "public_ip": "x.y.238.25",
>             "size": "24",
>             "source_nat": true,
>             "vif_mac_address": "06:d5:4e:00:00:1d"
>         }
>     ],
>     "id": "ips"
> ```
> Pinging [~wilder.rodrigues]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message