cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Francois Scheurer (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CLOUDSTACK-10043) Egress Rule in VPC ACL broken
Date Mon, 14 Aug 2017 07:45:01 GMT

    [ https://issues.apache.org/jira/browse/CLOUDSTACK-10043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16125343#comment-16125343
] 

Francois Scheurer commented on CLOUDSTACK-10043:
------------------------------------------------

Hello Rafael


Yes all VR were recreated.

We are using the 4.6.0 Version: root@r-1950-VM:~# cat /etc/cloudstack-release 
Cloudstack Release 4.6.0 Mon Dec 19 09:59:35 UTC 2016
We used the link http://packages.shapeblue.com/systemvmtemplate/4.6/new/systemvm64template-4.6-xen.vhd.bz2
from the docu here http://docs.cloudstack.apache.org/projects/cloudstack-release-notes/en/4.9.2.0/upgrade/upgrade-4.5.html
.
Do you think we should use another image?


We created the Rules via the ACS API.

Some of them were then present in the iptables filter table and some in mangle.
AFAIK all should be in filter and none in mangle.
And most importantly, the precedence numbers (in DB: network_acl_item_item.number) are not
respected in the VR.

Below are the rules from DB and from the Iptables to let you compare.
I will create a new VPC Offering and Network Offering from scratch and then create a new VPC
with some egress rules and post the results.


Best Regards
Francois Scheurer



ACL:
    mysql> select * from network_acl_item left join network_acl_item_cidrs on network_acl_item.id=network_acl_item_cidrs.network_acl_item_id
where network_acl_item.state="Active" and acl_id=(select network_acl_id from networks where
id=388);
    +------+--------------------------------------+--------+------------+----------+--------+----------+---------------------+-----------+-----------+--------------+--------+--------+---------+------+---------------------+-------------------+
    | id   | uuid                                 | acl_id | start_port | end_port | state
 | protocol | created             | icmp_code | icmp_type | traffic_type | number | action
| display | id   | network_acl_item_id | cidr              |
    +------+--------------------------------------+--------+------------+----------+--------+----------+---------------------+-----------+-----------+--------------+--------+--------+---------+------+---------------------+-------------------+
    | 2620 | 99cf33db-64fc-4708-be2b-fdc150300645 |    140 |        123 |      123 | Active
| udp      | 2016-04-06 19:29:27 |      NULL |      NULL | Egress       |     40 | Allow 
|       1 | 2988 |                2620 | AAA.BBB.111.18/32  |
    | 2596 | f06f7fdb-e68f-48c3-bbfa-1223c88e2f7b |    140 |         80 |       80 | Active
| tcp      | 2016-04-06 07:43:49 |      NULL |      NULL | Egress       |     50 | Allow 
|       1 | 2980 |                2596 | AAA.BBB.55.80/32    |
    | 2600 | afefcab3-89db-41d4-9ebe-89680f70416b |    140 |         80 |       80 | Active
| tcp      | 2016-04-06 07:44:42 |      NULL |      NULL | Egress       |     51 | Allow 
|       1 | 2956 |                2600 | AAA.BBB.9.40/32   |
    | 2604 | e65628b6-f050-4e17-b1f9-dbaba8a29e7c |    140 |         80 |       80 | Active
| tcp      | 2016-04-06 07:45:13 |      NULL |      NULL | Egress       |     52 | Allow 
|       1 | 2984 |                2604 | AAA.BBB.118.125/32 |
    | 2608 | 4b900d29-9eb7-4c7f-a2ff-15a94d35d2b3 |    140 |         80 |       80 | Active
| tcp      | 2016-04-06 07:45:51 |      NULL |      NULL | Egress       |     53 | Allow 
|       1 | 2968 |                2608 | AAA.BBB.55.80/32    |
    | 2612 | b38940e4-96b7-4f4b-b581-f0a8d5651e7a |    140 |         80 |       80 | Active
| tcp      | 2016-04-06 07:46:46 |      NULL |      NULL | Egress       |     54 | Allow 
|       1 | 2972 |                2612 | AAA.BBB.79.66/32  |
    | 2616 | 9f87246c-097c-414a-8320-e68905db6b95 |    140 |         80 |       80 | Active
| tcp      | 2016-04-06 07:47:51 |      NULL |      NULL | Egress       |     55 | Allow 
|       1 | 2976 |                2616 | AAA.BBB.116.80/32 |
    | 1044 | 09e83ceb-b2a8-4b1e-99e9-233b8449ac30 |    140 |        443 |      443 | Active
| tcp      | 2015-11-09 14:23:49 |      NULL |      NULL | Egress       |    100 | Allow 
|       1 | 1072 |                1044 | AAA.BBB.134.1/32  |
    | 1064 | 2e3c12b1-3222-4b27-956e-0811a157256b |    140 |        443 |      443 | Active
| tcp      | 2015-11-09 14:42:44 |      NULL |      NULL | Egress       |    110 | Allow 
|       1 | 1092 |                1064 | AAA.BBB.134.2/32  |
    | 1068 | 67084ffe-f419-4de8-b157-c92d4a2f8c6f |    140 |       3128 |     3128 | Active
| tcp      | 2015-11-09 14:47:20 |      NULL |      NULL | Ingress      |    150 | Allow 
|       1 | 1096 |                1068 | 0.0.0.0/0         |
    | 1048 | 7e44dc24-fba0-4e8e-9cd9-3aa26237e51f |    140 |         22 |       22 | Active
| tcp      | 2015-11-09 14:24:11 |      NULL |      NULL | Ingress      |    200 | Allow 
|       1 | 1076 |                1048 | AAA.BBB.117.0/25   |
    | 1052 | c866a5c8-b5ff-4c57-9644-dda5c703ca5f |    140 |         25 |       25 | Active
| tcp      | 2015-11-09 14:25:04 |      NULL |      NULL | Egress       |    300 | Allow 
|       1 | 1080 |                1052 | AAA.BBB.111.45/32  |
    | 1056 | 979114be-0b6f-44fc-9dd8-29ef93a77595 |    140 |       NULL |     NULL | Active
| icmp     | 2015-11-09 14:33:33 |         0 |         8 | Ingress      |    400 | Allow 
|       1 | 1084 |                1056 | 0.0.0.0/0         |
    | 1060 | d43a4d90-da14-476f-8a03-10c3d28817d2 |    140 |       NULL |     NULL | Active
| icmp     | 2015-11-09 14:34:01 |         4 |         3 | Ingress      |    500 | Allow 
|       1 | 1088 |                1060 | 0.0.0.0/0         |
    | 1500 | b05209b0-2dae-4ee7-8e37-632f7d843c03 |    140 |       5666 |     5666 | Active
| tcp      | 2015-12-10 17:11:43 |      NULL |      NULL | Ingress      |    600 | Allow 
|       1 | 1564 |                1500 | AAA.BBB.125.220/32 |
    | 1504 | 208c26ca-9563-4742-abae-6ed79b642177 |    140 |         22 |       22 | Active
| tcp      | 2015-12-10 17:12:03 |      NULL |      NULL | Ingress      |    700 | Allow 
|       1 | 1568 |                1504 | AAA.BBB.125.220/32 |
    | 1508 | 7b7f716f-86ba-4aae-a87c-06ee68c07a20 |    140 |       3128 |     3128 | Active
| tcp      | 2015-12-10 17:14:35 |      NULL |      NULL | Ingress      |    800 | Allow 
|       1 | 1572 |                1508 | AAA.BBB.125.220/32 |
    | 1188 | c4e6f89b-4b83-45aa-8cc3-1def7ad8c88d |    140 |        443 |      443 | Active
| tcp      | 2015-11-27 09:37:41 |      NULL |      NULL | Egress       |   1000 | Allow 
|       1 | 1252 |                1188 | AAA.BBB.71.201/32  |
    | 1637 | 7d6b7dd1-67da-4d7a-92b8-9b6f52757492 |    140 |         80 |       80 | Active
| tcp      | 2016-01-12 10:28:34 |      NULL |      NULL | Egress       |   1100 | Allow 
|       1 | 1713 |                1637 | AAA.BBB.111.13/32  |
    | 1876 | 6165879a-48af-48fa-9fee-85e0f6a670c1 |    140 |         80 |       80 | Active
| tcp      | 2016-02-25 09:02:41 |      NULL |      NULL | Egress       |   1200 | Allow 
|       1 | 2112 |                1876 | AAA.BBB.112.123/32 |
    | 1880 | 6a6571b7-ed32-432a-8cd5-07319ca53d60 |    140 |        290 |      291 | Active
| tcp      | 2016-02-25 09:03:16 |      NULL |      NULL | Ingress      |   1300 | Allow 
|       1 | 2116 |                1880 | AAA.BBB.112.123/32 |
    | 4155 | 13bfa058-2531-45af-9198-dbd86404ad07 |    140 |       NULL |     NULL | Active
| all      | 2017-08-08 08:05:19 |      NULL |      NULL | Egress       |   1400 | Allow 
|       1 | 4738 |                4155 | 0.0.0.0/0         |
    +------+--------------------------------------+--------+------------+----------+--------+----------+---------------------+-----------+-----------+--------------+--------+--------+---------+------+---------------------+-------------------+

Iptables with corresponding precedence number on the left (it is not an increasing serie as
it should be):
        # Generated by iptables-save v1.4.14 on Mon Aug  7 16:08:16 2017
    *mangle
        :PREROUTING ACCEPT [4:1184]
        :INPUT ACCEPT [0:0]
        :FORWARD ACCEPT [4:1184]
        :OUTPUT ACCEPT [0:0]
        :POSTROUTING ACCEPT [4:1184]
        :ACL_OUTBOUND_eth2 - [0:0]
        :VPN_STATS_eth1 - [0:0]
        -A PREROUTING -i eth2 -m state --state RELATED,ESTABLISHED -j CONNMARK --restore-mark
--nfmask 0xffffffff --ctmask 0xffffffff
        -A PREROUTING -s 10.4.96.0/27 ! -d 10.4.96.1/32 -i eth2 -m state --state NEW -j ACL_OUTBOUND_eth2
        -A PREROUTING -i eth1 -m state --state NEW -j CONNMARK --set-xmark 0x1/0xffffffff
        -A PREROUTING -s 10.4.96.9/32 -m state --state NEW -j MARK --set-xmark 0x1/0xffffffff
        -A PREROUTING -s 10.4.96.9/32 -m state --state NEW -j CONNMARK --save-mark --nfmask
0xffffffff --ctmask 0xffffffff
        -A FORWARD -j VPN_STATS_eth1
        -A OUTPUT -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
        -A POSTROUTING -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
        -A ACL_OUTBOUND_eth2 -d 224.0.0.18/32 -j ACCEPT
        -A ACL_OUTBOUND_eth2 -d 225.0.0.50/32 -j ACCEPT

    1400 -A ACL_OUTBOUND_eth2 -j DROP

    40 -A ACL_OUTBOUND_eth2 -d AAA.BBB.111.18/32 -p udp -m udp --dport 123 -j ACCEPT
    50 -A ACL_OUTBOUND_eth2 -d AAA.BBB.55.80/32 -p tcp -m tcp --dport 80 -j ACCEPT
    51 -A ACL_OUTBOUND_eth2 -d AAA.BBB.9.40/32 -p tcp -m tcp --dport 80 -j ACCEPT
    52 -A ACL_OUTBOUND_eth2 -d AAA.BBB.118.125/32 -p tcp -m tcp --dport 80 -j ACCEPT
    53 -A ACL_OUTBOUND_eth2 -d AAA.BBB.55.80/32 -p tcp -m tcp --dport 80 -j ACCEPT
    54 -A ACL_OUTBOUND_eth2 -d AAA.BBB.79.66/32 -p tcp -m tcp --dport 80 -j ACCEPT
    55 -A ACL_OUTBOUND_eth2 -d AAA.BBB.116.80/32 -p tcp -m tcp --dport 80 -j ACCEPT
    100 -A ACL_OUTBOUND_eth2 -d AAA.BBB.134.1/32 -p tcp -m tcp --dport 443 -j ACCEPT
    110 -A ACL_OUTBOUND_eth2 -d AAA.BBB.134.2/32 -p tcp -m tcp --dport 443 -j ACCEPT

    300 -A ACL_OUTBOUND_eth2 -d AAA.BBB.111.45/32 -p tcp -m tcp --dport 25 -j ACCEPT

    1000 -A ACL_OUTBOUND_eth2 -d AAA.BBB.71.201/32 -p tcp -m tcp --dport 443 -j ACCEPT
    1100 -A ACL_OUTBOUND_eth2 -d AAA.BBB.111.13/32 -p tcp -m tcp --dport 80 -j ACCEPT
    1200 -A ACL_OUTBOUND_eth2 -d AAA.BBB.112.123/32 -p tcp -m tcp --dport 80 -j ACCEPT

        -A VPN_STATS_eth1 -o eth1 -m mark --mark 0x525
        -A VPN_STATS_eth1 -i eth1 -m mark --mark 0x524
        COMMIT
        # Completed on Mon Aug  7 16:08:16 2017
        # Generated by iptables-save v1.4.14 on Mon Aug  7 16:08:16 2017
        *nat
        :PREROUTING ACCEPT [0:0]
        :INPUT ACCEPT [0:0]
        :OUTPUT ACCEPT [0:0]
        :POSTROUTING ACCEPT [0:0]
        -A PREROUTING -d 217.71.94.144/32 -i eth0 -j DNAT --to-destination 10.4.96.9
        -A PREROUTING -d 217.71.94.144/32 -j DNAT --to-destination 10.4.96.9
        -A OUTPUT -d 217.71.94.144/32 -j DNAT --to-destination 10.4.96.9
        -A POSTROUTING -s 10.4.96.0/27 -d 10.4.96.9/32 -o eth0 -j SNAT --to-source 10.4.96.1
        -A POSTROUTING -s 10.4.96.9/32 -o eth1 -j SNAT --to-source 217.71.94.144
        -A POSTROUTING -s 10.4.96.0/27 -o eth2 -j SNAT --to-source 10.4.96.1
        -A POSTROUTING -o eth1 -j SNAT --to-source 217.71.94.151
        COMMIT
        # Completed on Mon Aug  7 16:08:16 2017
        # Generated by iptables-save v1.4.14 on Mon Aug  7 16:08:16 2017
    *filter
        :INPUT DROP [0:0]
        :FORWARD DROP [0:0]
        :OUTPUT ACCEPT [0:0]
        :ACL_INBOUND_eth2 - [0:0]
        :FW_EGRESS_RULES - [0:0]
        :NETWORK_STATS - [0:0]
        :NETWORK_STATS_eth1 - [0:0]
        -A INPUT -d 10.4.96.1/32 -i eth2 -p tcp -m tcp --dport 443 -m state --state NEW -j
ACCEPT
        -A INPUT -d 10.4.96.1/32 -i eth2 -p tcp -m tcp --dport 80 -m state --state NEW -j
ACCEPT
        -A INPUT -d 10.4.96.1/32 -i eth2 -p tcp -m tcp --dport 53 -j ACCEPT
        -A INPUT -d 10.4.96.1/32 -i eth2 -p udp -m udp --dport 53 -j ACCEPT
        -A INPUT -j NETWORK_STATS
        -A INPUT -i eth2 -p udp -m udp --dport 67 -j ACCEPT
        -A INPUT -s 10.4.96.0/27 -i eth2 -p udp -m udp --dport 53 -j ACCEPT
        -A INPUT -s 10.4.96.0/27 -i eth2 -p tcp -m tcp --dport 53 -j ACCEPT
        -A INPUT -i eth2 -p tcp -m tcp --dport 80 -m state --state NEW -j ACCEPT
        -A INPUT -i eth2 -p tcp -m tcp --dport 8080 -m state --state NEW -j ACCEPT
        -A INPUT -d 224.0.0.18/32 -j ACCEPT
        -A INPUT -d 225.0.0.50/32 -j ACCEPT
        -A INPUT -p icmp -j ACCEPT
        -A INPUT -i lo -j ACCEPT
        -A INPUT -i eth0 -p tcp -m tcp --dport 3922 -m state --state NEW,ESTABLISHED -j ACCEPT
        -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
        -A FORWARD -j NETWORK_STATS_eth1
        -A FORWARD -j NETWORK_STATS
        -A FORWARD -s 10.4.96.0/24 ! -d 10.4.96.0/24 -j ACCEPT
        -A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT
        -A FORWARD -d 10.4.96.0/27 -o eth2 -j ACL_INBOUND_eth2
        -A FORWARD -d 10.4.96.9/32 -i eth1 -o eth0 -m state --state NEW -j ACCEPT
        -A OUTPUT -j NETWORK_STATS
        -A ACL_INBOUND_eth2 -d 225.0.0.50/32 -j ACCEPT
        -A ACL_INBOUND_eth2 -d 224.0.0.18/32 -j ACCEPT

    150 -A ACL_INBOUND_eth2 -p tcp -m tcp --dport 3128 -j ACCEPT
    200 -A ACL_INBOUND_eth2 -s AAA.BBB.117.0/25 -p tcp -m tcp --dport 22 -j ACCEPT

    400 -A ACL_INBOUND_eth2 -p icmp -m icmp --icmp-type 8/0 -j ACCEPT
    500 -A ACL_INBOUND_eth2 -p icmp -m icmp --icmp-type 3/4 -j ACCEPT
    600 -A ACL_INBOUND_eth2 -s AAA.BBB.125.220/32 -p tcp -m tcp --dport 5666 -j ACCEPT
    700 -A ACL_INBOUND_eth2 -s AAA.BBB.125.220/32 -p tcp -m tcp --dport 22 -j ACCEPT
    800 -A ACL_INBOUND_eth2 -s AAA.BBB.125.220/32 -p tcp -m tcp --dport 3128 -j ACCEPT

    1300 -A ACL_INBOUND_eth2 -s AAA.BBB.112.123/32 -p tcp -m tcp --dport 290:291 -j ACCEPT

        -A ACL_INBOUND_eth2 -j DROP
        -A NETWORK_STATS -i eth0 -o eth2 -p tcp
        -A NETWORK_STATS -i eth2 -o eth0 -p tcp
        -A NETWORK_STATS ! -i eth0 -o eth2 -p tcp
        -A NETWORK_STATS -i eth2 ! -o eth0 -p tcp
        -A NETWORK_STATS_eth1 -s 10.4.96.0/27 -o eth1
        -A NETWORK_STATS_eth1 -d 10.4.96.0/27 -i eth1
        COMMIT
        # Completed on Mon Aug  7 16:08:16 2017








>  Egress Rule in VPC ACL broken
> ------------------------------
>
>                 Key: CLOUDSTACK-10043
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10043
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: Virtual Router, VPC
>    Affects Versions: 4.9.2.0
>         Environment: CS 4.9.2 with XenServer 6.5SP1
>            Reporter: Francois Scheurer
>            Priority: Blocker
>
> The Network Offering of the VPC Tier has a Default Egress Policy = Deny.
> Some Allow Rules exist in the ACL, but _ALL_ egress connections are possible.
> Creating a Deny All rule explicit at the end of the rules is actually blocking ALL traffic
(should not, because of the Allow rules).
> The Iptables in the VR are wrong:
> 1)the allow rules are in wrong order.
> 2)some rules are in mangle table instead of filter
> Do you know how to fix this?
> Thank you for your help.
> Francois Scheurer



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message