Return-Path: X-Original-To: apmail-cloudstack-issues-archive@www.apache.org Delivered-To: apmail-cloudstack-issues-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 5267F10137 for ; Mon, 25 Nov 2013 13:11:37 +0000 (UTC) Received: (qmail 66509 invoked by uid 500); 25 Nov 2013 13:11:37 -0000 Delivered-To: apmail-cloudstack-issues-archive@cloudstack.apache.org Received: (qmail 66325 invoked by uid 500); 25 Nov 2013 13:11:36 -0000 Mailing-List: contact issues-help@cloudstack.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cloudstack.apache.org Delivered-To: mailing list issues@cloudstack.apache.org Received: (qmail 66310 invoked by uid 500); 25 Nov 2013 13:11:35 -0000 Delivered-To: apmail-incubator-cloudstack-issues@incubator.apache.org Received: (qmail 66306 invoked by uid 99); 25 Nov 2013 13:11:35 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 25 Nov 2013 13:11:35 +0000 Date: Mon, 25 Nov 2013 13:11:35 +0000 (UTC) From: "Anton Opgenoort (JIRA)" To: cloudstack-issues@incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (CLOUDSTACK-5256) ip mangle table is incorrect after stop/start of routerVM MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 Anton Opgenoort created CLOUDSTACK-5256: ------------------------------------------- Summary: ip mangle table is incorrect after stop/start of routerVM Key: CLOUDSTACK-5256 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5256 Project: CloudStack Issue Type: Bug Security Level: Public (Anyone can view this level - this is the default.) Components: Virtual Router Affects Versions: 4.1.0 Reporter: Anton Opgenoort When a routerVM is stopped and started again, the "-s " is not part of the iptables-save output, effectively removing the firewall restriction for that configured port. To reproduce: -create new network (default snat with conservative mode on) -start new VM on this new network, enable httpd -create port forward to new VM port 80 -create fw rule to selectively allow port 80 --> This works as expected, only trusted IP can connect to port 80. -now use the API to stop the routerVM, and start it up again. -->now port 80 is open to the world -to fix: add a bogus firewall rule (e.g. 1.1.1.1/32 port 81 allowed) -->now the port80 firewall rule has a source address again to filter. Details: (please check the difference every time in "-s 195.66.90.0/24" section of the mangle table) iptables BEFORE stop/start of router VM: root@r-15581-VM:~# iptables-save # Generated by iptables-save v1.4.14 on Mon Nov 25 12:29:51 2013 *nat :PREROUTING ACCEPT [206:14860] :INPUT ACCEPT [4:288] :OUTPUT ACCEPT [110:6742] :POSTROUTING ACCEPT [110:6742] -A PREROUTING -d 31.22.81.246/32 -i eth2 -p tcp -m tcp --dport 80 -j DNAT --to-destination 10.2.1.60:80 -A PREROUTING -d 31.22.81.246/32 -i eth0 -p tcp -m tcp --dport 80 -j DNAT --to-destination 10.2.1.60:80 -A OUTPUT -d 31.22.81.246/32 -p tcp -m tcp --dport 80 -j DNAT --to-destination 10.2.1.60:80 -A POSTROUTING -s 10.2.1.0/24 -d 10.2.1.60/32 -o eth0 -p tcp -m tcp --dport 80 -j SNAT --to-source 10.2.1.1 -A POSTROUTING -o eth2 -j SNAT --to-source 31.22.81.246 COMMIT # Completed on Mon Nov 25 12:29:51 2013 # Generated by iptables-save v1.4.14 on Mon Nov 25 12:29:51 2013 *mangle :PREROUTING ACCEPT [565:44756] :INPUT ACCEPT [369:30676] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [596:63294] :POSTROUTING ACCEPT [596:63294] :FIREWALL_31.22.81.246 - [0:0] :VPN_31.22.81.246 - [0:0] -A PREROUTING -d 31.22.81.246/32 -j VPN_31.22.81.246 -A PREROUTING -d 31.22.81.246/32 -j FIREWALL_31.22.81.246 -A PREROUTING -d 31.22.81.246/32 -i eth2 -p tcp -m tcp --dport 80 -j MARK --set-xmark 0x2/0xffffffff -A PREROUTING -d 31.22.81.246/32 -i eth2 -p tcp -m tcp --dport 80 -m state --state NEW -j CONNMARK --save-mark --nfmask 0xffffffff --ctmask 0xffffffff -A FIREWALL_31.22.81.246 -m state --state RELATED,ESTABLISHED -j ACCEPT -A FIREWALL_31.22.81.246 -s 195.66.90.0/24 -p tcp -m tcp --dport 80 -j RETURN -A FIREWALL_31.22.81.246 -j DROP -A VPN_31.22.81.246 -m state --state RELATED,ESTABLISHED -j ACCEPT -A VPN_31.22.81.246 -j RETURN COMMIT # Completed on Mon Nov 25 12:29:51 2013 # Generated by iptables-save v1.4.14 on Mon Nov 25 12:29:51 2013 *filter :INPUT ACCEPT [348:27584] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [533:53978] :NETWORK_STATS - [0:0] -A INPUT -j NETWORK_STATS -A FORWARD -j NETWORK_STATS -A FORWARD -d 10.2.1.60/32 -p tcp -m state --state RELATED,ESTABLISHED -m comment --comment "31.22.81.246:80:80" -j ACCEPT -A FORWARD -d 10.2.1.60/32 -p tcp -m tcp --dport 80 -m state --state NEW -m comment --comment "31.22.81.246:80:80" -j ACCEPT -A FORWARD -i eth2 -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT -A OUTPUT -j NETWORK_STATS -A NETWORK_STATS -i eth0 -o eth2 -A NETWORK_STATS -i eth2 -o eth0 -A NETWORK_STATS ! -i eth0 -o eth2 -p tcp -A NETWORK_STATS -i eth2 ! -o eth0 -p tcp COMMIT # Completed on Mon Nov 25 12:29:51 2013 root@r-15581-VM:~# iptables AFTER stop/start of router VM, see the missing "-s 195.66.90.0/24": root@r-15581-VM:~# iptables-save # Generated by iptables-save v1.4.14 on Mon Nov 25 12:32:07 2013 *mangle :PREROUTING ACCEPT [139:19436] :INPUT ACCEPT [135:19236] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [197:31830] :POSTROUTING ACCEPT [197:31830] :FIREWALL_31.22.81.246 - [0:0] :VPN_31.22.81.246 - [0:0] -A PREROUTING -d 31.22.81.246/32 -j VPN_31.22.81.246 -A PREROUTING -d 31.22.81.246/32 -j FIREWALL_31.22.81.246 -A PREROUTING -d 31.22.81.246/32 -i eth2 -p tcp -m tcp --dport 80 -j MARK --set-xmark 0x2/0xffffffff -A PREROUTING -d 31.22.81.246/32 -i eth2 -p tcp -m tcp --dport 80 -m state --state NEW -j CONNMARK --save-mark --nfmask 0xffffffff --ctmask 0xffffffff -A FIREWALL_31.22.81.246 -m state --state RELATED,ESTABLISHED -j ACCEPT -A FIREWALL_31.22.81.246 -p tcp -m tcp --dport 80 -j RETURN -A FIREWALL_31.22.81.246 -j DROP -A VPN_31.22.81.246 -m state --state RELATED,ESTABLISHED -j ACCEPT -A VPN_31.22.81.246 -j RETURN COMMIT # Completed on Mon Nov 25 12:32:07 2013 # Generated by iptables-save v1.4.14 on Mon Nov 25 12:32:07 2013 *nat :PREROUTING ACCEPT [12:712] :INPUT ACCEPT [6:360] :OUTPUT ACCEPT [4:240] :POSTROUTING ACCEPT [4:240] -A PREROUTING -d 31.22.81.246/32 -i eth2 -p tcp -m tcp --dport 80 -j DNAT --to-destination 10.2.1.60:80 -A PREROUTING -d 31.22.81.246/32 -i eth0 -p tcp -m tcp --dport 80 -j DNAT --to-destination 10.2.1.60:80 -A OUTPUT -d 31.22.81.246/32 -p tcp -m tcp --dport 80 -j DNAT --to-destination 10.2.1.60:80 -A POSTROUTING -o eth2 -j SNAT --to-source 31.22.81.246 -A POSTROUTING -s 10.2.1.0/24 -d 10.2.1.60/32 -o eth0 -p tcp -m tcp --dport 80 -j SNAT --to-source 10.2.1.1 COMMIT # Completed on Mon Nov 25 12:32:07 2013 # Generated by iptables-save v1.4.14 on Mon Nov 25 12:32:07 2013 *filter :INPUT ACCEPT [135:19236] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [169:26814] :NETWORK_STATS - [0:0] -A INPUT -j NETWORK_STATS -A FORWARD -j NETWORK_STATS -A FORWARD -i eth2 -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT -A FORWARD -d 10.2.1.60/32 -p tcp -m state --state RELATED,ESTABLISHED -m comment --comment "31.22.81.246:80:80" -j ACCEPT -A FORWARD -d 10.2.1.60/32 -p tcp -m tcp --dport 80 -m state --state NEW -m comment --comment "31.22.81.246:80:80" -j ACCEPT -A OUTPUT -j NETWORK_STATS -A NETWORK_STATS -i eth0 -o eth2 -A NETWORK_STATS -i eth2 -o eth0 -A NETWORK_STATS ! -i eth0 -o eth2 -p tcp -A NETWORK_STATS -i eth2 ! -o eth0 -p tcp COMMIT # Completed on Mon Nov 25 12:32:07 2013 root@r-15581-VM:~# -to fix: just add a bogus firewall rule to port 81, and all rules are re-applied, but correctly this time ("-s 195.66.90.0/24" has re-appeared again in the listing): root@r-15581-VM:/etc/iptables# iptables-save # Generated by iptables-save v1.4.14 on Mon Nov 25 12:37:23 2013 *mangle :PREROUTING ACCEPT [122:9848] :INPUT ACCEPT [70:6956] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [224:26638] :POSTROUTING ACCEPT [224:26638] :FIREWALL_31.22.81.246 - [0:0] :VPN_31.22.81.246 - [0:0] -A PREROUTING -d 31.22.81.246/32 -j VPN_31.22.81.246 -A PREROUTING -d 31.22.81.246/32 -j FIREWALL_31.22.81.246 -A PREROUTING -d 31.22.81.246/32 -i eth2 -p tcp -m tcp --dport 80 -j MARK --set-xmark 0x2/0xffffffff -A PREROUTING -d 31.22.81.246/32 -i eth2 -p tcp -m tcp --dport 80 -m state --state NEW -j CONNMARK --save-mark --nfmask 0xffffffff --ctmask 0xffffffff -A FIREWALL_31.22.81.246 -m state --state RELATED,ESTABLISHED -j ACCEPT -A FIREWALL_31.22.81.246 -s 195.66.90.0/24 -p tcp -m tcp --dport 80 -j RETURN -A FIREWALL_31.22.81.246 -s 1.1.1.1/32 -p tcp -m tcp --dport 81 -j RETURN -A FIREWALL_31.22.81.246 -j DROP -A VPN_31.22.81.246 -m state --state RELATED,ESTABLISHED -j ACCEPT -A VPN_31.22.81.246 -j RETURN COMMIT # Completed on Mon Nov 25 12:37:23 2013 # Generated by iptables-save v1.4.14 on Mon Nov 25 12:37:23 2013 *nat :PREROUTING ACCEPT [50:2784] :INPUT ACCEPT [0:0] :OUTPUT ACCEPT [32:1920] :POSTROUTING ACCEPT [32:1920] -A PREROUTING -d 31.22.81.246/32 -i eth2 -p tcp -m tcp --dport 80 -j DNAT --to-destination 10.2.1.60:80 -A PREROUTING -d 31.22.81.246/32 -i eth0 -p tcp -m tcp --dport 80 -j DNAT --to-destination 10.2.1.60:80 -A OUTPUT -d 31.22.81.246/32 -p tcp -m tcp --dport 80 -j DNAT --to-destination 10.2.1.60:80 -A POSTROUTING -s 10.2.1.0/24 -d 10.2.1.60/32 -o eth0 -p tcp -m tcp --dport 80 -j SNAT --to-source 10.2.1.1 -A POSTROUTING -o eth2 -j SNAT --to-source 31.22.81.246 COMMIT # Completed on Mon Nov 25 12:37:23 2013 # Generated by iptables-save v1.4.14 on Mon Nov 25 12:37:23 2013 *filter :INPUT ACCEPT [113:7156] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [227:25442] :NETWORK_STATS - [0:0] -A INPUT -j NETWORK_STATS -A FORWARD -j NETWORK_STATS -A FORWARD -d 10.2.1.60/32 -p tcp -m state --state RELATED,ESTABLISHED -m comment --comment "31.22.81.246:80:80" -j ACCEPT -A FORWARD -d 10.2.1.60/32 -p tcp -m tcp --dport 80 -m state --state NEW -m comment --comment "31.22.81.246:80:80" -j ACCEPT -A FORWARD -i eth2 -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT -A OUTPUT -j NETWORK_STATS -A NETWORK_STATS -i eth0 -o eth2 -A NETWORK_STATS -i eth2 -o eth0 -A NETWORK_STATS ! -i eth0 -o eth2 -p tcp -A NETWORK_STATS -i eth2 ! -o eth0 -p tcp COMMIT # Completed on Mon Nov 25 12:37:23 2013 root@r-15581-VM:/etc/iptables# -- This message was sent by Atlassian JIRA (v6.1#6144)