Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 39809200D2F for ; Wed, 1 Nov 2017 11:13:08 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 37EAB160BE6; Wed, 1 Nov 2017 10:13:08 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 07F80160BEA for ; Wed, 1 Nov 2017 11:13:06 +0100 (CET) Received: (qmail 46824 invoked by uid 500); 1 Nov 2017 10:13:06 -0000 Mailing-List: contact issues-help@cloudstack.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cloudstack.apache.org Delivered-To: mailing list issues@cloudstack.apache.org Received: (qmail 46815 invoked by uid 500); 1 Nov 2017 10:13:06 -0000 Delivered-To: apmail-incubator-cloudstack-issues@incubator.apache.org Received: (qmail 46812 invoked by uid 99); 1 Nov 2017 10:13:06 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Nov 2017 10:13:06 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 52EA71830DC for ; Wed, 1 Nov 2017 10:13:05 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -99.202 X-Spam-Level: X-Spam-Status: No, score=-99.202 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id VcKbun-9UbVi for ; Wed, 1 Nov 2017 10:13:02 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 4AF705FCC3 for ; Wed, 1 Nov 2017 10:13:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 7E07CE0D2B for ; Wed, 1 Nov 2017 10:13:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 2B5C9212F6 for ; Wed, 1 Nov 2017 10:13:00 +0000 (UTC) Date: Wed, 1 Nov 2017 10:13:00 +0000 (UTC) From: "Sven Vogel (JIRA)" To: cloudstack-issues@incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (CLOUDSTACK-10127) 4.9 / 4.10 KVM + openvswitch + vpc + static nat / secondary ip on eth2? MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Wed, 01 Nov 2017 10:13:08 -0000 [ https://issues.apache.org/jira/browse/CLOUDSTACK-10127?page=3Dcom.at= lassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sven Vogel updated CLOUDSTACK-10127: ------------------------------------ Description:=20 We have the following Problem.=20 1. KVM 2. Bridges bond with two interfaces and trunk (0,129,180,100-1500) to cloudbr0 Cloudbr0 (0 - guest network) Fakebridge pub129 (public network) Fakebridge sto180 (secondary storage network) Fakebridge mgmt0 (management) If I have a vpc all things work until I add a secondary ip and add a sta= tic nat. The following will happen, first address will be on the the correct inte= rface 146.0.122.134/26 but static nat will be on the false network. Its on the eth2=E2=80=A6 {{ root@r-29-VM:~# ip a 1: lo: mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo 2: eth0: mtu 1500 qdisc pfifo_fast sta= te UP qlen 1000 link/ether 0e:00:a9:fe:03:81 brd ff:ff:ff:ff:ff:ff inet 169.254.3.129/16 brd 169.254.255.255 scope global eth0 3: eth1: mtu 1500 qdisc pfifo_fast sta= te UP qlen 1000 link/ether 1e:00:2c:00:00:68 brd ff:ff:ff:ff:ff:ff inet 146.0.122.134/26 brd 146.0.122.191 scope global eth1 4: eth2: mtu 1500 qdisc pfifo_fast sta= te UP qlen 1000 link/ether 02:00:57:07:00:0c brd ff:ff:ff:ff:ff:ff inet 192.168.1.254/24 brd 192.168.1.255 scope global eth2 inet 146.0.122.135/26 brd 146.0.122.191 scope global eth2}} Normally I think the secondary ip should be on signed to eth1 not eth2! It sets my ip on the guest network vlan range on my cloudbr0 but it should = be pub129. vnet6 has 1353 guest tag and not the public tag. [root@kvm01 ~]# ovs-vsctl list-br cloud0 cloudbr0 mgmt0 pub129 sto180 [root@kvm01 ~]# virsh domiflist r-29-VM Interface Type Source Model MAC ------------------------------------------------------- vnet4 bridge cloud0 virtio 0e:00:a9:fe:03:81 vnet5 bridge pub129 virtio 1e:00:2c:00:00:68 vnet6 bridge cloudbr0 virtio 02:00:57:07:00:0c Bridge "cloud0" Port "vnet4" Interface "vnet4" Port "vnet5" tag: 129 Interface "vnet5" Port "vnet6" tag: 1353 Interface "vnet6" root@r-29-VM:~# cat /etc/cloudstack/ips.json { "eth0": [ { "add": true, "broadcast": "169.254.255.255", "cidr": "169.254.3.129/16", "device": "eth0", "gateway": "None", "netmask": "255.255.0.0", "network": "169.254.0.0/16", "nic_dev_id": "0", "nw_type": "control", "one_to_one_nat": false, "public_ip": "169.254.3.129", "size": "16", "source_nat": false } ], "eth1": [ { "add": true, "broadcast": "146.0.122.191", "cidr": "146.0.122.134/26", "device": "eth1", "first_i_p": true, "gateway": "146.0.122.130", "netmask": "255.255.255.192", "network": "146.0.122.128/26", "new_nic": false, "nic_dev_id": 1, "nw_type": "public", "one_to_one_nat": false, "public_ip": "146.0.122.134", "size": "26", "source_nat": true, "vif_mac_address": "1e:00:2c:00:00:68" } ], "eth2": [ { "add": true, "broadcast": "146.0.122.191", "cidr": "146.0.122.135/26", "device": "eth2", "first_i_p": true, "gateway": "146.0.122.130", "netmask": "255.255.255.192", "network": "146.0.122.128/26", "new_nic": false, "nic_dev_id": 2, "nw_type": "public", "one_to_one_nat": true, "public_ip": "146.0.122.135", "size": "26", "source_nat": true, "vif_mac_address": "1e:00:2c:00:00:68" }, { "add": false, "broadcast": "146.0.122.191", "cidr": "146.0.122.136/26", "device": "eth2", "first_i_p": true, "gateway": "146.0.122.130", "netmask": "255.255.255.192", "network": "146.0.122.128/26", "new_nic": false, "nic_dev_id": 2, "nw_type": "public", "one_to_one_nat": true, "public_ip": "146.0.122.136", "size": "26", "source_nat": true, "vif_mac_address": "1e:00:2c:00:00:68" }, { "add": true, "broadcast": "192.168.1.255", "cidr": "192.168.1.254/24", "device": "eth2", "gateway": "192.168.1.254", "netmask": "255.255.255.0", "network": "192.168.1.0/24", "nic_dev_id": "2", "nw_type": "guest", "one_to_one_nat": false, "public_ip": "192.168.1.254", "size": "24", "source_nat": false } ], "id": "ips" } Frank Maximus from Nuage analysed the problem. {quote} That seems to be a bug in the lookup of the device number, in case of openv= switch. The config clearly sets device to eth2, while it should be eth1. More specifically: in LibvirtComputingResource.prepareNetworkElementCommand() The broadcastUriToNicNum map is filled depending on the VR nics. In openvswitch the guest bridge is used as is, so it overwrites the mapping= of public. This was not an issue until 4.6 as then VR was using the macaddress to do l= ookup, while now it is using the device number. Kind Regards, Frank{quote} I hope anyone can fix that fastly. was: We have the following Problem.=20 1. KVM 2. Bridges bond with two interfaces and trunk (0,129,180,100-1500) to cloudbr0 Cloudbr0 (0 - guest network) Fakebridge pub129 (public network) Fakebridge sto180 (secondary storage network) Fakebridge mgmt0 (management) If I have a vpc all things work until I add a secondary ip and add a sta= tic nat. The following will happen, first address will be on the the correct inte= rface 146.0.122.134/26 but static nat will be on the false network. Its on the eth2=E2=80=A6 root@r-29-VM:~# ip a 1: lo: mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo 2: eth0: mtu 1500 qdisc pfifo_fast sta= te UP qlen 1000 link/ether 0e:00:a9:fe:03:81 brd ff:ff:ff:ff:ff:ff inet 169.254.3.129/16 brd 169.254.255.255 scope global eth0 3: eth1: mtu 1500 qdisc pfifo_fast sta= te UP qlen 1000 link/ether 1e:00:2c:00:00:68 brd ff:ff:ff:ff:ff:ff inet 146.0.122.134/26 brd 146.0.122.191 scope global eth1 4: eth2: mtu 1500 qdisc pfifo_fast sta= te UP qlen 1000 link/ether 02:00:57:07:00:0c brd ff:ff:ff:ff:ff:ff inet 192.168.1.254/24 brd 192.168.1.255 scope global eth2 inet 146.0.122.135/26 brd 146.0.122.191 scope global eth2 Normally I think the secondary ip should be on signed to eth1 not eth2! It sets my ip on the guest network vlan range on my cloudbr0 but it should = be pub129. vnet6 has 1353 guest tag and not the public tag. [root@kvm01 ~]# ovs-vsctl list-br cloud0 cloudbr0 mgmt0 pub129 sto180 [root@kvm01 ~]# virsh domiflist r-29-VM Interface Type Source Model MAC ------------------------------------------------------- vnet4 bridge cloud0 virtio 0e:00:a9:fe:03:81 vnet5 bridge pub129 virtio 1e:00:2c:00:00:68 vnet6 bridge cloudbr0 virtio 02:00:57:07:00:0c Bridge "cloud0" Port "vnet4" Interface "vnet4" Port "vnet5" tag: 129 Interface "vnet5" Port "vnet6" tag: 1353 Interface "vnet6" root@r-29-VM:~# cat /etc/cloudstack/ips.json { "eth0": [ { "add": true, "broadcast": "169.254.255.255", "cidr": "169.254.3.129/16", "device": "eth0", "gateway": "None", "netmask": "255.255.0.0", "network": "169.254.0.0/16", "nic_dev_id": "0", "nw_type": "control", "one_to_one_nat": false, "public_ip": "169.254.3.129", "size": "16", "source_nat": false } ], "eth1": [ { "add": true, "broadcast": "146.0.122.191", "cidr": "146.0.122.134/26", "device": "eth1", "first_i_p": true, "gateway": "146.0.122.130", "netmask": "255.255.255.192", "network": "146.0.122.128/26", "new_nic": false, "nic_dev_id": 1, "nw_type": "public", "one_to_one_nat": false, "public_ip": "146.0.122.134", "size": "26", "source_nat": true, "vif_mac_address": "1e:00:2c:00:00:68" } ], "eth2": [ { "add": true, "broadcast": "146.0.122.191", "cidr": "146.0.122.135/26", "device": "eth2", "first_i_p": true, "gateway": "146.0.122.130", "netmask": "255.255.255.192", "network": "146.0.122.128/26", "new_nic": false, "nic_dev_id": 2, "nw_type": "public", "one_to_one_nat": true, "public_ip": "146.0.122.135", "size": "26", "source_nat": true, "vif_mac_address": "1e:00:2c:00:00:68" }, { "add": false, "broadcast": "146.0.122.191", "cidr": "146.0.122.136/26", "device": "eth2", "first_i_p": true, "gateway": "146.0.122.130", "netmask": "255.255.255.192", "network": "146.0.122.128/26", "new_nic": false, "nic_dev_id": 2, "nw_type": "public", "one_to_one_nat": true, "public_ip": "146.0.122.136", "size": "26", "source_nat": true, "vif_mac_address": "1e:00:2c:00:00:68" }, { "add": true, "broadcast": "192.168.1.255", "cidr": "192.168.1.254/24", "device": "eth2", "gateway": "192.168.1.254", "netmask": "255.255.255.0", "network": "192.168.1.0/24", "nic_dev_id": "2", "nw_type": "guest", "one_to_one_nat": false, "public_ip": "192.168.1.254", "size": "24", "source_nat": false } ], "id": "ips" } Frank Maximus from Nuage analysed the problem. {quote} That seems to be a bug in the lookup of the device number, in case of openv= switch. The config clearly sets device to eth2, while it should be eth1. More specifically: in LibvirtComputingResource.prepareNetworkElementCommand() The broadcastUriToNicNum map is filled depending on the VR nics. In openvswitch the guest bridge is used as is, so it overwrites the mapping= of public. This was not an issue until 4.6 as then VR was using the macaddress to do l= ookup, while now it is using the device number. Kind Regards, Frank{quote} I hope anyone can fix that fastly. > 4.9 / 4.10 KVM + openvswitch + vpc + static nat / secondary ip on eth2? > ------------------------------------------------------------------------ > > Key: CLOUDSTACK-10127 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-101= 27 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the defa= ult.)=20 > Components: VPC > Affects Versions: 4.7.0, 4.8.0, 4.9.0, 4.10.0.0 > Environment: CentOS 7.4.1708 + KVM + OpenvSwitch 2.3-2.8 > Reporter: Sven Vogel > Priority: Critical > > We have the following Problem.=20 > 1. KVM > 2. Bridges > bond with two interfaces and trunk (0,129,180,100-1500) to cloudbr0 > Cloudbr0 (0 - guest network) > Fakebridge pub129 (public network) > Fakebridge sto180 (secondary storage network) > Fakebridge mgmt0 (management) > If I have a vpc all things work until I add a secondary ip and add a s= tatic nat. > The following will happen, first address will be on the the correct in= terface 146.0.122.134/26 but static nat will be on the false network. > Its on the eth2=E2=80=A6 > {{ root@r-29-VM:~# ip a > 1: lo: mtu 16436 qdisc noqueue state UNKNOWN > link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 > inet 127.0.0.1/8 scope host lo > 2: eth0: mtu 1500 qdisc pfifo_fast s= tate UP qlen 1000 > link/ether 0e:00:a9:fe:03:81 brd ff:ff:ff:ff:ff:ff > inet 169.254.3.129/16 brd 169.254.255.255 scope global eth0 > 3: eth1: mtu 1500 qdisc pfifo_fast s= tate UP qlen 1000 > link/ether 1e:00:2c:00:00:68 brd ff:ff:ff:ff:ff:ff > inet 146.0.122.134/26 brd 146.0.122.191 scope global eth1 > 4: eth2: mtu 1500 qdisc pfifo_fast s= tate UP qlen 1000 > link/ether 02:00:57:07:00:0c brd ff:ff:ff:ff:ff:ff > inet 192.168.1.254/24 brd 192.168.1.255 scope global eth2 > inet 146.0.122.135/26 brd 146.0.122.191 scope global eth2}} > Normally I think the secondary ip should be on signed to eth1 not eth2! > It sets my ip on the guest network vlan range on my cloudbr0 but it shoul= d be pub129. vnet6 has 1353 guest tag and not the public tag. > [root@kvm01 ~]# ovs-vsctl list-br > cloud0 > cloudbr0 > mgmt0 > pub129 > sto180 > [root@kvm01 ~]# virsh domiflist r-29-VM > Interface Type Source Model MAC > ------------------------------------------------------- > vnet4 bridge cloud0 virtio 0e:00:a9:fe:03:81 > vnet5 bridge pub129 virtio 1e:00:2c:00:00:68 > vnet6 bridge cloudbr0 virtio 02:00:57:07:00:0c > Bridge "cloud0" > Port "vnet4" > Interface "vnet4" > Port "vnet5" > tag: 129 > Interface "vnet5" > Port "vnet6" > tag: 1353 > Interface "vnet6" > root@r-29-VM:~# cat /etc/cloudstack/ips.json { > "eth0": [ > { > "add": true, > "broadcast": "169.254.255.255", > "cidr": "169.254.3.129/16", > "device": "eth0", > "gateway": "None", > "netmask": "255.255.0.0", > "network": "169.254.0.0/16", > "nic_dev_id": "0", > "nw_type": "control", > "one_to_one_nat": false, > "public_ip": "169.254.3.129", > "size": "16", > "source_nat": false > } > ], > "eth1": [ > { > "add": true, > "broadcast": "146.0.122.191", > "cidr": "146.0.122.134/26", > "device": "eth1", > "first_i_p": true, > "gateway": "146.0.122.130", > "netmask": "255.255.255.192", > "network": "146.0.122.128/26", > "new_nic": false, > "nic_dev_id": 1, > "nw_type": "public", > "one_to_one_nat": false, > "public_ip": "146.0.122.134", > "size": "26", > "source_nat": true, > "vif_mac_address": "1e:00:2c:00:00:68" > } > ], > "eth2": [ > { > "add": true, > "broadcast": "146.0.122.191", > "cidr": "146.0.122.135/26", > "device": "eth2", > "first_i_p": true, > "gateway": "146.0.122.130", > "netmask": "255.255.255.192", > "network": "146.0.122.128/26", > "new_nic": false, > "nic_dev_id": 2, > "nw_type": "public", > "one_to_one_nat": true, > "public_ip": "146.0.122.135", > "size": "26", > "source_nat": true, > "vif_mac_address": "1e:00:2c:00:00:68" > }, > { > "add": false, > "broadcast": "146.0.122.191", > "cidr": "146.0.122.136/26", > "device": "eth2", > "first_i_p": true, > "gateway": "146.0.122.130", > "netmask": "255.255.255.192", > "network": "146.0.122.128/26", > "new_nic": false, > "nic_dev_id": 2, > "nw_type": "public", > "one_to_one_nat": true, > "public_ip": "146.0.122.136", > "size": "26", > "source_nat": true, > "vif_mac_address": "1e:00:2c:00:00:68" > }, > { > "add": true, > "broadcast": "192.168.1.255", > "cidr": "192.168.1.254/24", > "device": "eth2", > "gateway": "192.168.1.254", > "netmask": "255.255.255.0", > "network": "192.168.1.0/24", > "nic_dev_id": "2", > "nw_type": "guest", > "one_to_one_nat": false, > "public_ip": "192.168.1.254", > "size": "24", > "source_nat": false > } > ], > "id": "ips" > } > Frank Maximus from Nuage analysed the problem. > {quote} > That seems to be a bug in the lookup of the device number, in case of ope= nvswitch. > The config clearly sets device to eth2, while it should be eth1. > More specifically: > in LibvirtComputingResource.prepareNetworkElementCommand() > The broadcastUriToNicNum map is filled depending on the VR nics. > In openvswitch the guest bridge is used as is, so it overwrites the mappi= ng of public. > This was not an issue until 4.6 as then VR was using the macaddress to do= lookup, while now it is using the device number. > Kind Regards, > Frank{quote} > I hope anyone can fix that fastly. -- This message was sent by Atlassian JIRA (v6.4.14#64029)