Mailing-List: contact dev-help@cloudstack.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cloudstack.apache.org
From: Simon Weller <sweller@ena.com>
To: "dev@cloudstack.apache.org" <dev@cloudstack.apache.org>
Subject: =?Windows-1252?Q?Re:_Working_Site-to-Site_VPN_gets_disconnected_and_VPC_s?=
 =?Windows-1252?Q?eems_to_forgets_ACL=92s?=
Thread-Topic: =?Windows-1252?Q?Working_Site-to-Site_VPN_gets_disconnected_and_VPC_seems?=
 =?Windows-1252?Q?_to_forgets_ACL=92s?=
Thread-Index: AQHR40bc8LoOaECbtEKEEFxvFg++MKAj6jUAgAC4kgCAACRIGg==
Date: Fri, 22 Jul 2016 18:29:16 +0000
Message-ID: <BY2PR02MB20075BE6CE8F662139047C3BA90A0@BY2PR02MB2007.namprd02.prod.outlook.com>
References: <CAN9+EFsu7atFFUPRratzMDdD+0eEZWcQZ3nwOCmVHg-KuMNdWg@mail.gmail.com>
 <569A2D9A-820C-4866-A7B7-17A2835C2FC1@persistent.co.in>,<CAN9+EFsEqyH2QhR4GioXzabgC4t6ZpPMS5vS6ROz1+ML=oxpzw@mail.gmail.com>
In-Reply-To: <CAN9+EFsEqyH2QhR4GioXzabgC4t6ZpPMS5vS6ROz1+ML=oxpzw@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-messagesentrepresentingtype: 1
x-originating-ip: [96.4.0.206]
x-ms-office365-filtering-correlation-id: 3707fcf2-74e5-46c0-e78d-08d3b25e16b9
x-microsoft-exchange-diagnostics: 1;BY2PR02MB2005;6:by+wUnZyjIum3Pry1GkS5JqK+dB0F/eGIDN32b9VBj/O8N/5+hHLuTSb2vPNpS3vmz06lS2/EM0YNv5Z3WYHVjgyuIaGESLh/og9uwCsQiSlpNxvKMPIbL3Y2Pr64q4DHeg6mGtaRD3AnGUQd9DOEEthlPtWVSvVUmda4YenmALe1TCrGOswGk+IP8xZ5F5o0VPTTqHbB1wwvPMh8NAU0Xpi6h2UbSV7WeYDCPK6t1BfHjUyHrOvOpgSLZrbytf+eKoR1Wc2UgTucltHnaViY9lx4ghir8ADYKo/bMxLjxk=;5:U5RgLJ0joohNe/vRHVBs1/7lJszPgAT8SBal/hHIf5gqST8O3ikhuuHWVSsoAeP5hremEmF3nIv0l0oMYDuCSO8lHKQKDjpNipUj2LqYCBhM5f1MIXi0e0OmhS1ekQVDHBUW9YxBxwNHF+oAO20wyg==;24:UapcxFWHsmzZDYieAXHWfECWSZnVR+EuSAb01fng0xMPjxwHeYonBZVFnFX60s64KlGdcJTuCoIiY987G1fSgSQQYhD4aOUnUJwwG7A2zFY=;7:DR6NeOKXndv/lvJua6a30QC2tGPj2KZMqTLhhxDHnDdRghQ+oiLwDulp/0GPpmyaiNGwv6lJPpG8Gj9uZSh5LCcGf8IwYYIxH0oOfTN977tRI9smpdMT67/yGUIZQYd1+A80ExZIEQvk5M/FV6R6JaskiCSjviQy/zsIa1o9iR7pZTtg3eNnjFWl8bwekVAZvhc3Y4mvz8rH2q5UiUop6s87o74OZaPrc9a9nXLDS13vT9tCcwzNL6G1o9bbmvLJ
x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BY2PR02MB2005;
x-microsoft-antispam-prvs: <BY2PR02MB20058CE5355F2B02508C812EA90A0@BY2PR02MB2005.namprd02.prod.outlook.com>
x-exchange-antispam-report-test: UriScan:(158342451672863)(266236234300168)(211936372134217)(179696456005106)(145926492361056)(17755550239193);
x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046);SRVR:BY2PR02MB2005;BCL:0;PCL:0;RULEID:;SRVR:BY2PR02MB2005;
x-forefront-prvs: 0011612A55
x-forefront-antispam-report: SFV:NSPM;SFS:(10009020)(7916002)(377424004)(199003)(377454003)(504964003)(189002)(24454002)(38564003)(40764003)(189998001)(110136002)(9686002)(107886002)(5002640100001)(86362001)(19580405001)(2351001)(122556002)(586003)(92566002)(3846002)(19625215002)(77096005)(10400500002)(50986999)(101416001)(102836003)(6116002)(19617315012)(3900700001)(15975445007)(7696003)(2501003)(68736007)(54356999)(66066001)(76176999)(1730700003)(81166006)(99286002)(33656002)(2906002)(16236675004)(7906003)(7736002)(106116001)(106356001)(74316002)(3280700002)(19627405001)(230783001)(5003600100003)(18206015028)(87936001)(2950100001)(2900100001)(105586002)(81156014)(76576001)(7846002)(3660700001)(19580395003)(97736004)(8936002)(7099028);DIR:OUT;SFP:1101;SCL:1;SRVR:BY2PR02MB2005;H:BY2PR02MB2007.namprd02.prod.outlook.com;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en;
received-spf: None (protection.outlook.com: ena.com does not designate
 permitted sender hosts)
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: multipart/alternative;
	boundary="_000_BY2PR02MB20075BE6CE8F662139047C3BA90A0BY2PR02MB2007namp_"
MIME-Version: 1.0
X-OriginatorOrg: ena.com
X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Jul 2016 18:29:16.3460
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 6dc38cd4-4d4f-4826-9649-17854289d170
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY2PR02MB2005
X-ENA-MailScanner-Information: Please contact support@ena.com for more information
X-ENA-MailScanner-ID: 7B50914801A4.AD0B3
X-ENA-MailScanner: No viruses found
X-ENA-MailScanner-SpamCheck: not spam, SpamAssassin (not cached, score=-4.59,
	required 4, ALL_TRUSTED -2.20, BAYES_00 -3.20, DKIM_SIGNED 0.10,
	DKIM_VALID -0.10, DKIM_VALID_AU -0.20, HTML_MESSAGE 1.20,
	SPF_HELO_PASS -0.20, T_REMOTE_IMAGE 0.01)
X-ENA-MailScanner-From: sweller@ena.com
X-ENA-MailScanner-Watermark: 1469816961.00898@0l/SIzlmVfMg6AZFJcXIlw
archived-at: Fri, 22 Jul 2016 18:29:42 -0000

--_000_BY2PR02MB20075BE6CE8F662139047C3BA90A0BY2PR02MB2007namp_
Content-Type: text/plain; charset="Windows-1252"
Content-Transfer-Encoding: quoted-printable

Do you use private gateways as well in your VPC environment? If so, does th=
e same ACL problem occur there as well?


________________________________
From: Jonas Schlichtenbrede <jonas.schlichtenbrede@gmail.com>
Sent: Friday, July 22, 2016 11:18 AM
To: dev@cloudstack.apache.org
Subject: Re: Working Site-to-Site VPN gets disconnected and VPC seems to fo=
rgets ACL=92s

Hi Jayapal,

thanks for you feedback!

We already tested the VPN with and without dead period detection - always
the same behaviour. I'll try 'ipsec auto --status' to see the output.

Browsing is just browsing to a website from within the VM (Windows Server
VM + IE). This works even if the VPN switches to disconnected...

At the moment we use NIC Bonding with the XenServers, but of course we
disabled one switch just to be sure that there is no general network or
switch issue. At the beginning we tested this setup without NIC Bonding,
too (again the same issues).

The strange thing is that everything is working for a few hours and then
just stops and a simple restart of the VPC Tier fixes it.

Is there a way to debug/analyse why the VR cuts/drops the connections or at
which stage (Xen, Switch, Top of the Rack Switch,...)?

Thanks
Jonas

PS: At guest networks we encountered that for example an active RDP session
(port forwardings in general) stopp working at the same time. Again
browsing to a website from a VM inside such a guest network is still
working...


On Fri, Jul 22, 2016 at 7:17 AM, Jayapal Uradi <jayapal.uradi@accelerite.co=
m
> wrote:

> Hi Jonas,
>
> It seems the connection is going down because the dead period detection.
>
> In router run the command 'ipsec auto =97status=92  to vpn connection  st=
atus.
> When the connection is down initiate traffic from the guest vm to other
> end of vpn and go to router check the ipsec vpn status (ipsec auto =97sta=
tus).
> This gives wether the connection is up or not in the VR.  It takes router
> status get interval to update the VPN status.
>
> The browsing you mentioned is about browsing the other end of vpn servers=
 ?
>
> Thanks,
> Jayapal
>
> > On Jul 21, 2016, at 5:25 PM, Jonas Schlichtenbrede <
> jonas.schlichtenbrede@gmail.com> wrote:
> >
> > Hi CloudStack Users and Developers,
> >
> > we=92re currently implementing a new CloudStack environment based on
> 4.8.0.1
> > (System VM Template is 4.6) with XenServer 6.5 SP1 and all the latest
> > updates.
> >
> > So far everything works as expected we only have an issue regarding the
> > stability of Site-to-Site VPNs within VPCs and we think ACL=92s.
> >
> > I=92ll try to describe the problem and behaviour:
> >
> > A connected and working S2S VPN switches to disconnected after some tim=
e
> > (usually a few hours). In relation to that the VPC seems to =93forget=
=94 it=92s
> > ACLs. Restarting only the Network Tier (a VM lives within) solves the
> > issues for a short period of time (1-3 hours). The state of the VPN
> > switches to connected and the S2S VPN is working again. Also pinging fr=
om
> > the VM to any public address is working again. Strange is, that for
> example
> > browsing to a website is working all the time. Isolated networks howeve=
r
> > work like a charm.
> >
> > We tried to solve this issue through several tests. We changing the
> network
> > setup and reducing the complexity just to get this behaviour isolated.
> But
> > it=92s always the same. We also tried several different connections to
> > different customer gateways (firewalls) and a VPC-VPN to VPC-VPN
> connection
> > to another CloudStack deployment (based on Version 4.5.2) without any
> > success.
> >
> > In addition, we tested several setups like CentOS 6 and CentOS 7, but
> again
> > always the same. We updated one installation to the master from yesterd=
ay
> > 4.9.0.0-snapshot =96 again no success. We do not have any issues with
> version
> > 4.5.2 =96 but this installation is in a different datacentre.
> >
> > Below you=92ll find some logs =96 the relevant IP for this test connect=
ion
> is:
> > *85.88.16.104*
> >
> > CloudStack 4.8.0.1 Logs (Google Docs):
> >
> >
> https://drive.google.com/open?id=3D1gqIjDdG1htps4p1t7m1uHSs7aNHplWp1Np83n=
H6e7zM
[https://lh4.googleusercontent.com/exnf2bX69PTpe-2SCy0IkHIPWjrlJX4t4KGprTaq=
iFH1C9pCF5QeiXZkmjOGQrO-E4MJ_Q=3Dw1200-h630-p]<https://drive.google.com/ope=
n?id=3D1gqIjDdG1htps4p1t7m1uHSs7aNHplWp1Np83nH6e7zM>

CloudStack - Management Server Logs<https://drive.google.com/open?id=3D1gqI=
jDdG1htps4p1t7m1uHSs7aNHplWp1Np83nH6e7zM>
drive.google.com
Working Site-to-Site VPN gets disconnected CloudStack Logs -- 19.07.2016 Li=
nks to full Cloudstack Logs CloudStack Logs -- 19.07.2016 CloudStack Logs f=
rom 2016-07-19 2016-07-19 21:29:54,505 DEBUG [c.c.a.t.Request] (RouterStatu=
sMonitor-1:ctx-6d6037bf) (logid:b1054a97) Seq 1-886167669178158...


> >
> >
> > IPsec Logs from the Virtual Router:
> >
> https://drive.google.com/open?id=3D1ZWvhFu2P_Wv_lF8TgYMmexeS_KDag1Mp-kmuh=
l8l7uU
> >
> >
> > Thank you in advance for your help!
> >
> > Jonas
> >
> > PS: If possible from your site we can do a remote session to take a loo=
k
> at
> > the setup.
>
>
>
>
> DISCLAIMER
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> This e-mail may contain privileged and confidential information which is
> the property of Accelerite, a Persistent Systems business. It is intended
> only for the use of the individual or entity to which it is addressed. If
> you are not the intended recipient, you are not authorized to read, retai=
n,
> copy, print, distribute or use this message. If you have received this
> communication in error, please notify the sender and delete all copies of
> this message. Accelerite, a Persistent Systems business does not accept a=
ny
> liability for virus infected mails.
>

--_000_BY2PR02MB20075BE6CE8F662139047C3BA90A0BY2PR02MB2007namp_--