Mailing-List: contact users-help@cloudstack.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@cloudstack.apache.org
MIME-Version: 1.0
In-Reply-To: <940185856.4522.1510296264699@ox.pcextreme.nl>
References: <1750999994.6369.1509476065042.JavaMail.zimbra@li.nux.ro>
 <CAMvtBPMCKB1KJeYw4a-tY_XHNYj+SR08v_1SEz5VsYQpzhyyuQ@mail.gmail.com>
 <270992313.1239.1509970237965.JavaMail.zimbra@li.nux.ro> <CAD97ioeHboORsFXbdLjQZ6rCH01Esbw+3+j4Q6tLamRuSafCZA@mail.gmail.com>
 <1778806379.4501.1510211582337@ox.pcextreme.nl> <CAMvtBPNrWQGphKyQixJQCPbRLDODzTx_Ct4YQ4zd8ia1gnHVGQ@mail.gmail.com>
 <1657929870.1138.1510253995466.JavaMail.zimbra@li.nux.ro> <940185856.4522.1510296264699@ox.pcextreme.nl>
From: Pierre-Luc Dion <pdion@cloudops.com>
Date: Fri, 10 Nov 2017 08:27:07 -0500
Message-ID: <CAD97iodfuN85Lkc5eLhBuusXWsZmqcy1w4Fzba3EY6mjsdb4Qw@mail.gmail.com>
Subject: Re: HTTPS LB and x-forwarded-for
To: Wido den Hollander <wido@widodh.nl>
Cc: "Nux!" <nux@li.nux.ro>, dev@cloudstack.apache.org,
	users <users@cloudstack.apache.org>, Khosrow Moossavi <kmoossavi@cloudops.com>,
	Will Stevens <wstevens@cloudops.com>
Content-Type: multipart/alternative; boundary="94eb2c07d580a6a73e055da0e0a4"
archived-at: Fri, 10 Nov 2017 13:27:20 -0000

--94eb2c07d580a6a73e055da0e0a4
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

I kind of like the proxy backend type, ill check on our end if that would
work but definitely a simple and efficient approach!


Le 10 nov. 2017 01 h 44, "Wido den Hollander" <wido@widodh.nl> a =C3=A9crit=
 :

>
> > Op 9 november 2017 om 19:59 schreef Nux! <nux@li.nux.ro>:
> >
> >
> > Wido,
> >
> > Excellent suggestion with the "transparent proxy", I was not aware of
> that.
> > I think that would be a great idea and wouldn't require too many
> modifications, especially as Haproxy comes already with the VR.
> >
>
> It's indeed just a matter of a HAProxy config setting. We could make it
> configurable per backend in HAProxy. Regular HTTP, TCP or PROXY for examp=
le.
>
> That way your problem would be solved.
>
> Wido
>
> > To Paul:
> > - imho the LB solution ACS ships now is a bit handicaped since you do
> not know the remote host ip. You're flying blind unless you use google
> analytics (and these things have gotten more and more aggressively filter=
ed
> by adblocks).
> > Enhancing Haproxy as Wido suggested would go a long way, it wouldn't
> break existing functionality and would also keep SSL processing off the V=
R.
> >
> > --
> > Sent from the Delta quadrant using Borg technology!
> >
> > Nux!
> > www.nux.ro
> >
> > ----- Original Message -----
> > > From: "Andrija Panic" <andrija.panic@gmail.com>
> > > To: "users" <users@cloudstack.apache.org>
> > > Cc: "Khosrow Moossavi" <kmoossavi@cloudops.com>, "Will Stevens" <
> wstevens@cloudops.com>, "dev"
> > > <dev@cloudstack.apache.org>, "Pierre-Luc Dion" <pdion@cloudops.com>
> > > Sent: Thursday, 9 November, 2017 13:10:58
> > > Subject: Re: HTTPS LB and x-forwarded-for
> >
> > > Wido,
> > >
> > > backend servers are not Linux only, for example we have a ton of
> Windows
> > > customers, some WEB solutions / IIS etc...
> > >
> > > @all - If we try to please/solve everyone's proxying
> solution/requirement -
> > > this is impossible IMHO - I'm thinking more about some "do it as you
> like"
> > > solution, to let customer write his own haproxy config and upoad it
> (for
> > > example, or something better?).
> > >
> > > We can support newer version of haproxy (1.5+) which also implement
> > > "transarent proxy" (integrate with kernel so to speak)  to allow
> TCP-level
> > > connections to backend (TCP mode, not HTTP mode) but to still
> "preserve"
> > > remote IP by faking it (fake soruce IP =3D transarent proxy).
> > >
> > > For the rest of configuration options,  I would leave it to the
> customer
> > > how he/she wants to configure rest of haproxy configuration, inlcudin=
g
> > > custom checks, etc. Haproxy configuration is never-ending story, and =
we
> > > probably should allow custom sripts/configuration instead of trying t=
o
> > > provide GUI/API way to configure everything (which is impossible...)
> > >
> > > Just my 2 cents...
> > >
> > > On 9 November 2017 at 08:13, Wido den Hollander <wido@widodh.nl>
> wrote:
> > >
> > >>
> > >> > Op 8 november 2017 om 14:59 schreef Pierre-Luc Dion <
> pdion@cloudops.com
> > >> >:
> > >> >
> > >> >
> > >> > Same challenge here too!
> > >> >
> > >> > Let's look at improving Load-balancing offering from cloudstack, I
> guest
> > >> we
> > >> > should do a feature spec draft soon..,  from my perspective, doing
> SSL
> > >> > offload on the VR could be problematic if the VR spec if too small=
,
> and
> > >> the
> > >> > default spec of the VR being 1vcpu@256MB, considering it can be th=
e
> > >> router
> > >> > of a VPC, doing VPN termination, adding HTTPS  is a bit ish... Wha=
t
> would
> > >> > be your thought about this ?
> > >> >
> > >> > I'd be curious to have a LB offering in ACS where it would deploy =
a
> > >> > redundant traefik[1] beside the VR for doing http and https
> > >> Load-balancing.
> > >> > I think it would also be useful if the API of that traefik instanc=
e
> would
> > >> > be available from within the VPC or LBnetwork so is API would be
> > >> accessible
> > >> > to other apps orchestration tools such as  kubernetes or rancher.
> > >> >
> > >> > traefik or not, here is what I think is needed by cloudstack in th=
e
> LB
> > >> > improvement:
> > >> >
> > >> > - support http, https (X-Forwarded-For)
> > >>
> > >> HAProxy also supports the PROXY protocol towards the backends. Apach=
e
> > >> 2.4.22 supports this natively and Varnish for example can also talk
> PROXY.
> > >>
> > >> It adds a littlebit of metadata to the connection so that the backen=
d
> > >> knows the original IP the connection came from for example:
> > >> https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt
> > >>
> > >> Wido
> > >>
> > >> > - basic persistence tuning (API already exist)
> > >> > - better backend monitoring, currently only a tcp connect validate
> if the
> > >> > webserver is up.
> > >> > - ssl offload
> > >> > - metric collection, more stats, maybe just export the tool status
> page
> > >> to
> > >> > the private network.
> > >> > - Container world support, right now if you have Rancher or
> kubernetes
> > >> > cluster, you need to deploy your own LB solution behing mostlikely=
 a
> > >> static
> > >> > nat., If cloudstack would deploy a traefik instance, Kub or Ranche=
r
> could
> > >> > reuse this instance and managed it to properly do LB between
> containers.
> > >> >
> > >> >
> > >> > What would be your prefered LB tool:
> > >> > haproxy, traefik or nginx?
> > >> >
> > >> > CloudStack already have to code to handle SSL certs per projects a=
nd
> > >> > accounts if not mistaking because that code was added to support
> > >> NetScaler
> > >> > as Load-balancer in the past. so one less thing to think about :-)
> > >> >
> > >> >
> > >> > [1] https://traefik.io/
> > >> >
> > >> >
> > >> > PL,
> > >> >
> > >> >
> > >> >
> > >> > On Mon, Nov 6, 2017 at 7:10 AM, Nux! <nux@li.nux.ro> wrote:
> > >> >
> > >> > > Thanks Andrija,
> > >> > >
> > >> > > LB outside of the VR sounds like a good idea. An appliance based
> on,
> > >> say
> > >> > > cloud-init + ansible and so on could do the trick; alas it'd nee=
d
> to be
> > >> > > outside ACS.
> > >> > > I guess as users we could maybe come up with a spec for an
> > >> improvement, at
> > >> > > least we'd have something the devs could look at whenever it is
> > >> possible.
> > >> > >
> > >> > > Regards,
> > >> > > Lucian
> > >> > >
> > >> > > --
> > >> > > Sent from the Delta quadrant using Borg technology!
> > >> > >
> > >> > > Nux!
> > >> > > www.nux.ro
> > >> > >
> > >> > > ----- Original Message -----
> > >> > > > From: "Andrija Panic" <andrija.panic@gmail.com>
> > >> > > > To: "dev" <dev@cloudstack.apache.org>
> > >> > > > Cc: "users" <users@cloudstack.apache.org>
> > >> > > > Sent: Thursday, 2 November, 2017 23:21:37
> > >> > > > Subject: Re: HTTPS LB and x-forwarded-for
> > >> > >
> > >> > > > We used to make some special stuff for one of the clients,
> where all
> > >> LB
> > >> > > > configuration work is done from outside of the ACS, i.e. pytho=
n
> > >> script to
> > >> > > > feed/configure VR - install latest haproxy 1.5.x for transpare=
nt
> > >> proxy,
> > >> > > > since client insisted on SSL termination done on backend web S=
SL
> > >> > > servers....
> > >> > > > Not good idea, that is all I can say (custom configuration
> thing) -
> > >> but
> > >> > > the
> > >> > > > LB setup is actually good - transparent mode haproxy, works on
> TCP
> > >> level,
> > >> > > > so you can see "real client IP" on the backend servers (which
> must
> > >> use VR
> > >> > > > as the default gtw, as per default, so the whole setup works
> > >> properly).
> > >> > > >
> > >> > > > I'm still looking forward to see some special support of LB
> inside
> > >> VR via
> > >> > > > ACS - proper LB setup inside VR via GUI/API -  i.e. to enable =
LB
> > >> > > > provisioning SCRIPT (bash, or whatever),  where all needed
> > >> > > > install+configure can be done from client side  - otherwise
> covering
> > >> all
> > >> > > > user cases, with proper HTTP checks and similar....is
> impossible to
> > >> do
> > >> > > > IMHO.
> > >> > > >
> > >> > > > Some other clients, actually have internal FW appliance (i.e.
> > >> multihomed
> > >> > > > VM, acting as gtw for all VMs in all networks), and haproxy
> instaled
> > >> on
> > >> > > > this device (with NAT configured from VR to this internal
> FW/VM, so
> > >> > > remote
> > >> > > > IP can be seen properly) - this setup is fully under customer
> > >> control,
> > >> > > and
> > >> > > > can provide any kind of special haproxy config...
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > On 31 October 2017 at 19:54, Nux! <nux@li.nux.ro> wrote:
> > >> > > >
> > >> > > >> Hello,
> > >> > > >>
> > >> > > >> Of the people running an LB (VR) with https backends, how do
> you
> > >> deal
> > >> > > with
> > >> > > >> the lack of x-forwarded-for since for port 443 there's just
> simple
> > >> TCP
> > >> > > >> balancing?
> > >> > > >>
> > >> > > >> Has anyone thought of terminating SSL in the VR instead? Idea=
s?
> > >> > > >>
> > >> > > >> Cheers
> > >> > > >>
> > >> > > >> --
> > >> > > >> Sent from the Delta quadrant using Borg technology!
> > >> > > >>
> > >> > > >> Nux!
> > >> > > >> www.nux.ro
> > >> > > >>
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > --
> > >> > > >
> > >> > > > Andrija Pani=C4=87
> > >> > >
> > >>
> > >
> > >
> > >
> > > --
> > >
> > > Andrija Pani=C4=87
>

--94eb2c07d580a6a73e055da0e0a4--