Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id D726B200CCC for ; Thu, 6 Jul 2017 12:34:15 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id D5C8E166042; Thu, 6 Jul 2017 10:34:15 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id CE890166029 for ; Thu, 6 Jul 2017 12:34:14 +0200 (CEST) Received: (qmail 6971 invoked by uid 500); 6 Jul 2017 10:34:11 -0000 Mailing-List: contact issues-help@cloudstack.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cloudstack.apache.org Delivered-To: mailing list issues@cloudstack.apache.org Received: (qmail 6542 invoked by uid 500); 6 Jul 2017 10:34:11 -0000 Delivered-To: apmail-incubator-cloudstack-issues@incubator.apache.org Received: (qmail 5344 invoked by uid 99); 6 Jul 2017 10:34:10 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 06 Jul 2017 10:34:10 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 9DD2AC456D for ; Thu, 6 Jul 2017 10:34:09 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -99.202 X-Spam-Level: X-Spam-Status: No, score=-99.202 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id nLNHrYdH5mI9 for ; Thu, 6 Jul 2017 10:34:06 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 2A0C160DC4 for ; Thu, 6 Jul 2017 10:34:06 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 2F385E0E08 for ; Thu, 6 Jul 2017 10:34:05 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id EEBB924675 for ; Thu, 6 Jul 2017 10:34:03 +0000 (UTC) Date: Thu, 6 Jul 2017 10:34:03 +0000 (UTC) From: "Rajani Karuturi (JIRA)" To: cloudstack-issues@incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (CLOUDSTACK-9940) Rules ( PF , Firewall , LB)when deleted during the VR stopped state are still persistent on the VR iptables. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Thu, 06 Jul 2017 10:34:16 -0000 [ https://issues.apache.org/jira/browse/CLOUDSTACK-9940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajani Karuturi updated CLOUDSTACK-9940: ---------------------------------------- Fix Version/s: (was: 4.10.0.0) 4.10.1.0 > Rules ( PF , Firewall , LB)when deleted during the VR stopped state are still persistent on the VR iptables. > ------------------------------------------------------------------------------------------------------------ > > Key: CLOUDSTACK-9940 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9940 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the default.) > Components: Virtual Router > Affects Versions: 4.10.0.0 > Reporter: DeepthiMachiraju > Labels: PVR > Fix For: 4.10.1.0 > > Attachments: cloud.log > > > - Create a isolated network and configure PF , FW , LB rules on the source Nat IP. > - Stop the VR from CS and delete the above rules , and add new rules with different port numbers. > - Start the VR from CS and check if the above rules are configured . > Observation : > - Rules which are newly added during the VR stop state are configured properly. > - Rules which are deleted are still retained in the respective json files and reflecting in the iptable rules. > - Rules which are deleted are cleaned up from the DB and UI , but still persistent in iptables. > ********************************************** > mysql> select * from port_forwarding_rules; > +----+-------------+-----------------+-----------------+---------------+ > | id | instance_id | dest_ip_address | dest_port_start | dest_port_end | > +----+-------------+-----------------+-----------------+---------------+ > | 38 | 4 | 172.16.1.227 | 22 | 22 | > | 51 | 23 | 10.1.1.18 | 888 | 888 | > +----+-------------+-----------------+-----------------+---------------+ > < forwardingrules.json > > - 2000 port is the one which was deleted when router is in stopped state. > - 888 port is the newly added rule when VR in stopped state . > root@r-29-VM:/etc/cloudstack# cat forwardingrules.json > { > "10.147.52.21": [ > { > "internal_ip": "10.1.1.18", > "internal_ports": "2000:2000", > "protocol": "tcp", > "public_ip": "10.147.52.21", > "public_ports": "2000:2000", > "type": "forward" > }, > { > "internal_ip": "10.1.1.18", > "internal_ports": "888:888", > "protocol": "tcp", > "public_ip": "10.147.52.21", > "public_ports": "888:888", > "type": "forward" > } > ], > "id": "forwardingrules" > ****************************************************** > Firewall Rules : > mysql> select * from firewall_rules where network_id=209 and purpose='Firewall'; > +----+--------------------------------------+---------------+------------+----------+--------+----------+----------+------------+-----------+------------+--------------------------------------+---------------------+-----------+-----------+---------+------+--------+--------------+---------+ > | id | uuid | ip_address_id | start_port | end_port | state | protocol | purpose | account_id | domain_id | network_id | xid | created | icmp_code | icmp_type | related | type | vpc_id | traffic_type | display | > +----+--------------------------------------+---------------+------------+----------+--------+----------+----------+------------+-----------+------------+--------------------------------------+---------------------+-----------+-----------+---------+------+--------+--------------+---------+ > | 50 | a41a75b3-ba8b-4126-b098-f52fa8151891 | 12 | 8888 | 8888 | Active | tcp | Firewall | 2 | 1 | 209 | e608b208-6e27-41c4-9163-40f3f3829929 | 2017-06-05 10:29:02 | NULL | NULL | NULL | User | NULL | Ingress | 1 | > +----+--------------------------------------+---------------+------------+----------+--------+----------+----------+------------+-----------+------------+--------------------------------------+---------------------+-----------+-----------+---------+------+--------+--------------+---------+ > 1 row in set (0.00 sec) > < firewallrules.json > > - 555 port was deleted when VR in stopped state . > - 8888 port was added when VR in stopped state > root@r-29-VM:/etc/cloudstack# cat firewallrules.json > { > "0": { > "already_added": false, > "default_egress_policy": false, > "id": 0, > "protocol": "all", > "purpose": "Firewall", > "revoked": false, > "source_cidr_list": [], > "src_ip": "", > "traffic_type": "Egress" > }, > "49": { > "already_added": false, > "default_egress_policy": false, > "id": 49, > "protocol": "tcp", > "purpose": "Firewall", > "revoked": false, > "source_cidr_list": [ > "1.1.1.1/32" > ], > "src_ip": "10.147.52.21", > "src_port_range": [ > 555, > 555 > ], > "traffic_type": "Ingress" > }, > "50": { > "already_added": true, > "default_egress_policy": false, > "id": 50, > "protocol": "tcp", > "purpose": "Firewall", > "revoked": false, > "source_cidr_list": [ > "2.2.2.0/24" > ], > "src_ip": "10.147.52.21", > "src_port_range": [ > 8888, > 8888 > ], > "traffic_type": "Ingress" > }, > "id": "firewallrules" > ************************************************************** > root@r-29-VM:/etc/cloudstack# iptables -t nat -L > Chain PREROUTING (policy ACCEPT) > target prot opt source destination > DNAT tcp -- anywhere 10.147.52.21 tcp dpt:cisco-sccp to:10.1.1.18:2000 > DNAT tcp -- anywhere 10.147.52.21 tcp dpt:cisco-sccp to:10.1.1.18:2000 > MARK tcp -- anywhere 10.147.52.21 tcp dpt:cisco-sccp MARK set 0x2 > CONNMARK tcp -- anywhere 10.147.52.21 tcp dpt:cisco-sccp state NEW CONNMARK save > DNAT tcp -- anywhere 10.147.52.21 tcp dpt:888 to:10.1.1.18:888 > DNAT tcp -- anywhere 10.147.52.21 tcp dpt:888 to:10.1.1.18:888 > MARK tcp -- anywhere 10.147.52.21 tcp dpt:888 MARK set 0x2 > CONNMARK tcp -- anywhere 10.147.52.21 tcp dpt:888 state NEW CONNMARK save > Chain INPUT (policy ACCEPT) > target prot opt source destination > Chain OUTPUT (policy ACCEPT) > target prot opt source destination > DNAT tcp -- anywhere 10.147.52.21 tcp dpt:cisco-sccp to:10.1.1.18:2000 > DNAT tcp -- anywhere 10.147.52.21 tcp dpt:888 to:10.1.1.18:888 > Chain POSTROUTING (policy ACCEPT) > target prot opt source destination > SNAT all -- anywhere anywhere to:10.147.52.21 > SNAT tcp -- 10.1.1.0/24 v22 tcp dpt:cisco-sccp to:10.1.1.1 > SNAT tcp -- 10.1.1.0/24 v22 tcp dpt:888 to:10.1.1.1 > *********************************************************** > 2017-06-05 06:33:12,633 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-192:ctx-0deed038) (logid:e11c0ab3) VR Config file VR-e16a066b-d50e-48d2-9ba2-e582afd185e5.cfg got created in VR, ip 169.254.1.97 with content > #Apache CloudStack Virtual Router Config File > > 1.0 > > > /var/cache/cloud/ip_associations.json > {"ip_address":[{"public_ip":"10.147.52.21","source_nat":true,"add":true,"one_to_one_nat":false,"first_i_p":true,"gateway":"10.147.52.1","netmask":"255.255.255.0","vif_mac_address":"06:45:88:00:00:17","nic_dev_id":2,"new_nic":false,"nw_type":"Public"}],"type":"ips"} > > > > /var/cache/cloud/firewall_rules.json > {"rules":[{"id":50,"src_ip":"10.147.52.21","protocol":"tcp","src_port_range":[8888,8888],"revoked":false,"already_added":true,"source_cidr_list":["2.2.2.0/24"],"purpose":"Firewall","traffic_type":"Ingress","default_egress_policy":false}],"type":"firewallrules"} > > > > /var/cache/cloud/forwarding_rules.json > {"rules":[{"revoke":false,"protocol":"tcp","source_ip_address":"10.147.52.21","source_port_range":"888:888","destination_ip_address":"10.1.1.18","destination_port_range":"888:888"}],"type":"forwardrules"} > > > > /var/cache/cloud/load_balancer.json > {"rules":[{"configuration":["global","\tlog 127.0.0.1:3914 local0 warning","\tmaxconn 4096","\tmaxpipes 1024","\tchroot /var/lib/haproxy","\tuser haproxy","\tgroup haproxy","\tdaemon","\t ","defaults","\tlog global","\tmode tcp","\toption dontlognull","\tretries 3","\toption redispatch","\toption forwardfor","\toption forceclose","\ttimeout connect 5000","\ttimeout client 50000","\ttimeout server 50000","\nlisten stats_on_public 10.147.52.21:8081\n\tmode http\n\toption httpclose\n\tstats enable\n\tstats uri /admin?stats\n\tstats realm Haproxy\\ Statistics\n\tstats auth admin1:AdMiN123\n","\t ","listen 10_147_52_21-666 10.147.52.21:666","\tbalance roundrobin","\tserver 10_147_52_21-666_0 10.1.1.18:666 check","\t ","\t "],"tmp_cfg_file_path":"/etc/haproxy/","tmp_cfg_file_name":"haproxy.cfg.new.1496658791133","add_rules":["10.147.52.21:666:"],"remove_rules":[],"stat_rules":["10.147.52.21:8081:0/0:,"],"router_ip":"169.254.1.97"}],"type":"loadbalancer"} > > > > /var/cache/cloud/monitor_service.json > {"config":"[dhcp]:processname=dnsmasq:servicename=dnsmasq:pidfile=/var/run/dnsmasq/dnsmasq.pid:,[loadbalancing]:processname=haproxy:servicename=haproxy:pidfile=/var/run/haproxy.pid:,[ssh]:processname=sshd:servicename=ssh:pidfile=/var/run/sshd.pid:,[webserver]:processname=apache2:servicename=apache2:pidfile=/var/run/apache2.pid:,","type":"monitorservice"} > > > > /var/cache/cloud/vm_dhcp_entry.json > {"host_name":"v11","mac_address":"02:00:2b:1f:00:01","ipv4_adress":"10.1.1.101","ipv6_duid":"00:03:00:01:02:00:2b:1f:00:01","dns_adresses":"10.1.1.1","default_gateway":"10.1.1.1","default_entry":true,"type":"dhcpentry"} > > > > /var/cache/cloud/vm_dhcp_entry.json > {"host_name":"v22","mac_address":"02:00:5f:59:00:04","ipv4_adress":"10.1.1.18","ipv6_duid":"00:03:00:01:02:00:5f:59:00:04","dns_adresses":"10.1.1.1","default_gateway":"10.1.1.1","default_entry":true,"type":"dhcpentry"} > > > > /var/cache/cloud/vm_metadata.json > {"vm_ip_address":"10.1.1.101","vm_metadata":[["userdata","user-data",null],["metadata","service-offering","Small Instance"],["metadata","availability-zone","z1"],["metadata","local-ipv4","10.1.1.101"],["metadata","local-hostname","v11"],["metadata","public-ipv4","10.147.52.21"],["metadata","public-hostname","10.147.52.21"],["metadata","instance-id","d09cf9fe-cddc-4f8a-952f-5dadede3ab91"],["metadata","vm-id","d09cf9fe-cddc-4f8a-952f-5dadede3ab91"],["metadata","public-keys",null],["metadata","cloud-identifier","CloudStack-{5a63cc81-c2c7-46ac-a959-bff4d5b12f19}"]],"type":"vmdata"} > > > > /var/cache/cloud/vm_metadata.json > {"vm_ip_address":"10.1.1.18","vm_metadata":[["userdata","user-data",null],["metadata","service-offering","Small Instance"],["metadata","availability-zone","z1"],["metadata","local-ipv4","10.1.1.18"],["metadata","local-hostname","v22"],["metadata","public-ipv4","10.147.52.21"],["metadata","public-hostname","10.147.52.21"],["metadata","instance-id","7f524b97-6e2f-4b19-b40e-6da7f59f3f2e"],["metadata","vm-id","7f524b97-6e2f-4b19-b40e-6da7f59f3f2e"],["metadata","public-keys",null],["metadata","cloud-identifier","CloudStack-{5a63cc81-c2c7-46ac-a959-bff4d5b12f19}"]],"type":"vmdata"} > > > 2017-06-05 06:33:12,634 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-192:ctx-0deed038) (logid:e11c0ab3) Executing command in VR: /opt/cloud/bin/router_proxy.sh vr_cfg.sh 169.254.1.97 -c /var/cache/cloud/VR-e16a066b-d50e-48d2-9ba2-e582afd185e5.cfg > 2017-06-05 06:33:13,059 DEBUG [c.c.a.ApiServlet] (catalina-exec-2:ctx-0bd038a0) (logid:d85c5105) ===START=== 10.233.89.32 -- GET command=queryAsyncJobResult&jobId=e11c0ab3-55fb-4a30-a0fd-dc550b1b45f5&response=json&_=1496658792996 > 2017-06-05 06:33:13,142 DEBUG [c.c.a.ApiServlet] (catalina-exec-2:ctx-0bd038a0 ctx-7be6be07) (logid:d85c5105) ===END=== 10.233.89.32 -- GET command=queryAsyncJobResult&jobId=e11c0ab3-55fb-4a30-a0fd-dc550b1b45f5&response=json&_=1496658792996 > 2017-06-05 06:33:13,225 DEBUG [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-6:null) (logid:) SeqA 2-144522: Processing Seq 2-144522: { Cmd , MgmtId: -1, via: 2, Ver: v1, Flags: 11, [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":2,"_loadInfo":"{\n \"connections\": []\n}","wait":0}}] } > 2017-06-05 06:33:13,231 DEBUG [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-6:null) (logid:) SeqA 2-144522: Sending Seq 2-144522: { Ans: , MgmtId: 6760647622781, via: 2, Ver: v1, Flags: 100010, [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] } > 2017-06-05 06:33:14,554 DEBUG [c.c.s.StatsCollector] (StatsCollector-6:ctx-ca3f95b2) (logid:27ed7e15) VmStatsCollector is running... > 2017-06-05 06:33:14,599 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-13:ctx-42ce21d1) (logid:ee6b66c9) Seq 1-2918895508489535017: Executing request > ******************************************************************************* > Attached cloud.log -- This message was sent by Atlassian JIRA (v6.4.14#64029)