cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Serg Senko <kernc...@gmail.com>
Subject Re: Cloudstack 4.3 instances can't access outside world
Date Mon, 21 Apr 2014 05:39:23 GMT
Hi,

I have same issue after upgrade 4.1.1 to 4.3.0
Take a look, in CS4.2 VR you have NIC's eth0,eth1,eth2.
In CS 4.3 VR you have 4 NIC's where the eth2 and eth3 is the same.

How CS4.3 is passed the QA?



On Sat, Apr 12, 2014 at 12:16 AM, motty cruz <motty.cruz@gmail.com> wrote:

> I have a testing cloudstack cluster, I destroyed it and rebuilding upgraded
> serveral times, each time I ran into the same problem, unable to access
> outside world from instances behind virtual router.
>
> here is iptables before upgrade, Cloudstack 4.2
>
> # Generated by iptables-save v1.4.14 on Fri Apr 11 19:53:57 2014
> *mangle
> :PREROUTING ACCEPT [2317:1282555]
> :INPUT ACCEPT [409:147015]
> :FORWARD ACCEPT [0:0]
> :OUTPUT ACCEPT [189:29312]
> :POSTROUTING ACCEPT [189:29312]
> :FIREWALL_176.23.23.192 - [0:0]
> :VPN_176.23.23.192 - [0:0]
> -A PREROUTING -d 176.23.23.192/32 -j VPN_176.23.23.192
> -A PREROUTING -d 176.23.23.192/32 -j FIREWALL_176.23.23.192
> -A PREROUTING -m state --state RELATED,ESTABLISHED -j CONNMARK
> --restore-mark --nfmask 0xffffffff --ctmask 0xffffffff
> -A POSTROUTING -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
> -A FIREWALL_176.23.23.192 -m state --state RELATED,ESTABLISHED -j ACCEPT
> -A FIREWALL_176.23.23.192 -j DROP
> -A VPN_176.23.23.192 -m state --state RELATED,ESTABLISHED -j ACCEPT
> -A VPN_176.23.23.192 -j RETURN
> COMMIT
> # Completed on Fri Apr 11 19:53:57 2014
> # Generated by iptables-save v1.4.14 on Fri Apr 11 19:53:57 2014
> *filter
> :INPUT DROP [204:117504]
> :FORWARD DROP [0:0]
> :OUTPUT ACCEPT [150:22404]
> :FW_OUTBOUND - [0:0]
> :NETWORK_STATS - [0:0]
> -A INPUT -j NETWORK_STATS
> -A INPUT -d 224.0.0.18/32 -j ACCEPT
> -A INPUT -d 225.0.0.50/32 -j ACCEPT
> -A INPUT -i eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT
> -A INPUT -i eth1 -m state --state RELATED,ESTABLISHED -j ACCEPT
> -A INPUT -i eth2 -m state --state RELATED,ESTABLISHED -j ACCEPT
> -A INPUT -p icmp -j ACCEPT
> -A INPUT -i lo -j ACCEPT
> -A INPUT -i eth0 -p udp -m udp --dport 67 -j ACCEPT
> -A INPUT -i eth0 -p udp -m udp --dport 53 -j ACCEPT
> -A INPUT -i eth0 -p tcp -m tcp --dport 53 -j ACCEPT
> -A INPUT -i eth1 -p tcp -m state --state NEW -m tcp --dport 3922 -j ACCEPT
> -A INPUT -i eth0 -p tcp -m state --state NEW -m tcp --dport 80 -j ACCEPT
> -A INPUT -s 10.1.1.0/24 -i eth0 -p tcp -m state --state NEW -m tcp --dport
> 8080 -j ACCEPT
> -A FORWARD -j NETWORK_STATS
> -A FORWARD -i eth0 -o eth1 -m state --state RELATED,ESTABLISHED -j ACCEPT
> -A FORWARD -i eth0 -o eth0 -m state --state NEW -j ACCEPT
> -A FORWARD -i eth0 -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT
> -A FORWARD -i eth2 -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT
> -A FORWARD -i eth0 -o eth2 -j FW_OUTBOUND
> -A OUTPUT -j NETWORK_STATS
> -A FW_OUTBOUND -m state --state RELATED,ESTABLISHED -j ACCEPT
> -A NETWORK_STATS -i eth0 -o eth2
> -A NETWORK_STATS -i eth2 -o eth0
> -A NETWORK_STATS ! -i eth0 -o eth2 -p tcp
> -A NETWORK_STATS -i eth2 ! -o eth0 -p tcp
> COMMIT
> # Completed on Fri Apr 11 19:53:57 2014
> # Generated by iptables-save v1.4.14 on Fri Apr 11 19:53:57 2014
> *nat
> :PREROUTING ACCEPT [2078:1204416]
> :INPUT ACCEPT [10:964]
> :OUTPUT ACCEPT [1:338]
> :POSTROUTING ACCEPT [1:338]
> -A POSTROUTING -o eth2 -j SNAT --to-source 176.23.23.192
> COMMIT
> # Completed on Fri Apr 11 19:53:57 2014
>
>
> after upgrading to Cloudstack 4.3
>
>
> :POSTROUTING ACCEPT [211:25828]
> :FIREWALL_176.23.23.192 - [0:0]
> :VPN_176.23.23.192 - [0:0]
> -A PREROUTING -d 176.23.23.192/32 -j VPN_176.23.23.192
> -A PREROUTING -d 176.23.23.192/32 -j FIREWALL_176.23.23.192
> -A PREROUTING -m state --state RELATED,ESTABLISHED -j CONNMARK
> --restore-mark --nfmask 0xffffffff --ctmask 0xffffffff
> -A POSTROUTING -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
> -A FIREWALL_176.23.23.192 -m state --state RELATED,ESTABLISHED -j ACCEPT
> -A FIREWALL_176.23.23.192 -j DROP
> -A VPN_176.23.23.192 -m state --state RELATED,ESTABLISHED -j ACCEPT
> -A VPN_176.23.23.192 -j RETURN
> COMMIT
> # Completed on Fri Apr 11 20:49:46 2014
> # Generated by iptables-save v1.4.14 on Fri Apr 11 20:49:46 2014
> *filter
> :INPUT DROP [68:32168]
> :FORWARD DROP [0:0]
> :OUTPUT ACCEPT [81:12516]
> :FW_EGRESS_RULES - [0:0]
> :FW_OUTBOUND - [0:0]
> :NETWORK_STATS - [0:0]
> -A INPUT -j NETWORK_STATS
> -A INPUT -d 224.0.0.18/32 -j ACCEPT
> -A INPUT -d 225.0.0.50/32 -j ACCEPT
> -A INPUT -i eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT
> -A INPUT -i eth1 -m state --state RELATED,ESTABLISHED -j ACCEPT
> -A INPUT -i eth2 -m state --state RELATED,ESTABLISHED -j ACCEPT
> -A INPUT -p icmp -j ACCEPT
> -A INPUT -i lo -j ACCEPT
> -A INPUT -i eth0 -p udp -m udp --dport 67 -j ACCEPT
> -A INPUT -i eth0 -p udp -m udp --dport 53 -j ACCEPT
> -A INPUT -i eth0 -p tcp -m tcp --dport 53 -j ACCEPT
> -A INPUT -i eth1 -p tcp -m state --state NEW -m tcp --dport 3922 -j ACCEPT
> -A INPUT -i eth0 -p tcp -m state --state NEW -m tcp --dport 80 -j ACCEPT
> -j ACCEPT
> -A FORWARD -j NETWORK_STATS
> -A FORWARD -i eth0 -o eth1 -m state --state RELATED,ESTABLISHED -j ACCEPT
> -A FORWARD -i eth2 -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT
> -A FORWARD -i eth0 -o eth0 -m state --state NEW -j ACCEPT
> -A FORWARD -i eth0 -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT
> -A FORWARD -i eth0 -o eth2 -j FW_OUTBOUND
> -A FORWARD -i eth3 -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT
> -A FORWARD -i eth0 -o eth3 -j FW_OUTBOUND
> -A OUTPUT -j NETWORK_STATS
> -A FW_EGRESS_RULES -p udp -m udp --dport 1:65535 -j ACCEPT
> -A FW_EGRESS_RULES -p icmp -m icmp --icmp-type 8/0 -j ACCEPT
> -A FW_EGRESS_RULES -p tcp -m tcp --dport 1:65535 -j ACCEPT
> -A FW_OUTBOUND -m state --state RELATED,ESTABLISHED -j ACCEPT
> -A FW_OUTBOUND -j FW_EGRESS_RULES
> -A NETWORK_STATS -i eth0 -o eth2
> -A NETWORK_STATS -i eth2 -o eth0
> -A NETWORK_STATS ! -i eth0 -o eth2 -p tcp
> -A NETWORK_STATS -i eth2 ! -o eth0 -p tcp
> -A NETWORK_STATS -i eth0 -o eth3
> -A NETWORK_STATS -i eth3 -o eth0
> -A NETWORK_STATS ! -i eth0 -o eth3 -p tcp
> -A NETWORK_STATS -i eth3 ! -o eth0 -p tcp
> COMMIT
> # Completed on Fri Apr 11 20:49:46 2014
> # Generated by iptables-save v1.4.14 on Fri Apr 11 20:49:46 2014
> *nat
> :PREROUTING ACCEPT [685:441108]
> :INPUT ACCEPT [7:492]
> :OUTPUT ACCEPT [2:136]
> :POSTROUTING ACCEPT [3:412]
> -A POSTROUTING -o eth3 -j SNAT --to-source 199.96.39.192
> COMMIT
> # Completed on Fri Apr 11 20:49:46 2014
>
>
> Please help!! is a KVM cluster, I have two bridges, cloudbr0 and cloudbr1.
> bridge cloudbr0 is default public/guest and cloudbr1 is management.
>
> I am able to ping from virtual router, I personally think source NAT is the
> issue,
>
> Thanks,
>
>
> On Fri, Apr 11, 2014 at 8:14 AM, motty cruz <motty.cruz@gmail.com> wrote:
>
> > looking at the VR it has three interfaces NIC1 10.1.1.1 NIC2 (type
> > control) Local Link NIC3(default NIC) public IP
> >
> > when I log in to the router and ran the following command:
> > ifconfig -a
> > I see 4 interfaces eth0 10.1.1.1 eth1 (local link)169.254.3.81 eth2
> public
> > IP eth3 (same public IP as eth2).
> >
> > I believe this is my problem, but don't know how to fix it,
> > any ideas?
> >
> >
> > On Fri, Apr 11, 2014 at 8:04 AM, motty cruz <motty.cruz@gmail.com>
> wrote:
> >
> >> Thanks Suresh,
> >>
> >> I tried your suggestion and I'm not able to access outside the VR
> router.
> >> I am stumped!
> >>
> >> please help!
> >>
> >>
> >>
> >>
> >> On Thu, Apr 10, 2014 at 7:21 AM, Suresh Sadhu <Suresh.Sadhu@citrix.com
> >wrote:
> >>
> >>> Ok  then work around is manually append rule to cloudbr1 .
> >>>
> >>> Take the backup of iptables rules
> >>> Manfully detach the eth interface from  cloudbr0  and attach to
> cloudbr1
> >>> Apply the all exiting  firewall  rules manually on the interface gain
> >>>
> >>>
> >>> After that your VMs will access the public network.
> >>>
> >>>
> >>> Regards
> >>> Sadhu
> >>>
> >>>
> >>>
> >>> -----Original Message-----
> >>> From: motty cruz [mailto:motty.cruz@gmail.com]
> >>> Sent: 10 April 2014 19:40
> >>> To: users@cloudstack.apache.org
> >>> Subject: Re: Cloudstack 4.3 instances can't access outside world
> >>>
> >>> yes, I'm am using traffic labels, everything was working fine before
> the
> >>> upgrade to 4.3. did not change anything on the cloudbr0 or cloudbr1.
> >>>
> >>>
> >>> On Thu, Apr 10, 2014 at 7:05 AM, Suresh Sadhu <Suresh.Sadhu@citrix.com
> >>> >wrote:
> >>>
> >>> > Did you used traffic name labels?
> >>> >
> >>> > In 4.3 traffic labels are not considering ,by default its attaching
> to
> >>> > default  traffic labels(eg:in KVM its cloudbr0 ...due to this unable
> >>> > to access public network i.r before upgrade if ieth2 attached
> cloudbr1
> >>> > and after upgrade its attached to cloudbr0).maybe you are hitting
> this
> >>> issue.
> >>> >
> >>> > Regards
> >>> > sadhu
> >>> >
> >>> >
> >>> > -----Original Message-----
> >>> > From: motty cruz [mailto:motty.cruz@gmail.com]
> >>> > Sent: 10 April 2014 19:28
> >>> > To: users@cloudstack.apache.org
> >>> > Subject: Re: Cloudstack 4.3 instances can't access outside world
> >>> >
> >>> > yes I can ping VR, also after the upgrade VR has four insterfaces,
> >>> > eth0 subnet for Instances, eth1, eth2 for public IP and eth3 for
> >>> public IP.
> >>> >
> >>> >
> >>> > On Wed, Apr 9, 2014 at 10:35 PM, Erik Weber <terbolous@gmail.com>
> >>> wrote:
> >>> >
> >>> > > Can you ping the VR? Log on to the VR, and get the iptables rules.
> >>> > > How do they look?
> >>> > >
> >>> > > Erik Weber
> >>> > > 10. apr. 2014 00:21 skrev "motty cruz" <motty.cruz@gmail.com>
> >>> følgende:
> >>> > >
> >>> > > > I did add egress rules, reboot network but no sucess, so
I
> removed
> >>> > > > that rules and nothing.
> >>> > > >
> >>> > > > I am lost.
> >>> > > >
> >>> > > >
> >>> > > > On Wed, Apr 9, 2014 at 9:08 AM, Erik Weber <terbolous@gmail.com>
> >>> > wrote:
> >>> > > >
> >>> > > > > Did you remove the egress rule again? If not, try that.
> >>> > > > >
> >>> > > > > Erik
> >>> > > > > 9. apr. 2014 15:49 skrev "motty cruz" <motty.cruz@gmail.com>
> >>> > følgende:
> >>> > > > >
> >>> > > > > > yes I try adding the rule, restart network and
router but no
> >>> > success!
> >>> > > > > >
> >>> > > > > >
> >>> > > > > > On Tue, Apr 8, 2014 at 11:16 PM, Erik Weber
> >>> > > > > > <terbolous@gmail.com>
> >>> > > > wrote:
> >>> > > > > >
> >>> > > > > > > Try adding an egress rule, and removing it
again.
> >>> > > > > > >
> >>> > > > > > > We experience the same, but has so far believed
it was
> >>> > > > > > > because we
> >>> > > > > changed
> >>> > > > > > > the default rule from deny to allow after
accounts were
> >>> made..
> >>> > > > > > >
> >>> > > > > > >
> >>> > > > > > > On Tue, Apr 8, 2014 at 11:14 PM, motty cruz
> >>> > > > > > > <motty.cruz@gmail.com>
> >>> > > > > > wrote:
> >>> > > > > > >
> >>> > > > > > > > I have two isolated network both virtual
routers can ping
> >>> > > anywhere,
> >>> > > > > but
> >>> > > > > > > the
> >>> > > > > > > > Instances behind the virtual router can't
ping or access
> >>> > > > > > > > the
> >>> > > > > internet.
> >>> > > > > > > >
> >>> > > > > > > >
> >>> > > > > > > >
> >>> > > > > > > >
> >>> > > > > > > > On Tue, Apr 8, 2014 at 10:38 AM, motty
cruz <
> >>> > > motty.cruz@gmail.com>
> >>> > > > > > > wrote:
> >>> > > > > > > >
> >>> > > > > > > > > Hello,
> >>> > > > > > > > > I'm having issues with VMs unable
to access outside
> >>> world.
> >>> > > > > > > > > I
> >>> > > can
> >>> > > > > ping
> >>> > > > > > > > > gateway, also when I log in to virtual
router, I am
> able
> >>> > > > > > > > > to
> >>> > > ping
> >>> > > > > > > > > google.com or anywhere.
> >>> > > > > > > > > in the Egress rules I am allowing
all. reboot network
> >>> > > > > > > > > and
> >>> > > virtual
> >>> > > > > > > router
> >>> > > > > > > > > does not help.
> >>> > > > > > > > >
> >>> > > > > > > > > VMs were able to access outside
before upgrading from
> >>> > > > > > > > > 4.2 to
> >>> > > 4.3.
> >>> > > > > > > > >
> >>> > > > > > > > > any ideas?
> >>> > > > > > > > >
> >>> > > > > > > >
> >>> > > > > > >
> >>> > > > > >
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> >>
> >>
> >
>



-- 
ttyv0 "/usr/libexec/gmail Pc"  webcons on secure

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message