Mailing-List: contact users-help@trafficserver.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@trafficserver.apache.org
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\))
Subject: Re: Migrating from squid
From: Leif Hedstrom <zwoop@apache.org>
In-Reply-To: 
 <CABXB=RQgR2=CFgwB+kH9_sCbbfhz0C0+97kYiJn7RAkX-mytDg@mail.gmail.com>
Date: Fri, 27 Feb 2015 10:50:41 -0500
Content-Transfer-Encoding: quoted-printable
Message-Id: <3D01BE33-22BE-46EF-A070-701B5920A08A@apache.org>
References: 
 <CABXB=RQgR2=CFgwB+kH9_sCbbfhz0C0+97kYiJn7RAkX-mytDg@mail.gmail.com>
To: users@trafficserver.apache.org,
 J David <j.david.lists@gmail.com>


> On Feb 27, 2015, at 12:51 AM, J David <j.david.lists@gmail.com> wrote:
>=20
> (Not sure if this is best for the user list or the dev list, as it's
> development, but not of ATS per se.)
>=20
> Squid offers a feature called url_rewrite_program that can be used to
> change its behavior based on client IP, requested URL, and various
> other criteria.  It runs a bunch of copies of an external program and
> passes information to and from them over pipes.
>=20
> Overall, ATS is a much better fit for our environment, but we are
> heavily dependent on this feature of Squid.
>=20
> The short version of what we need to do is this:
>=20
> Based on a (ClientIP,RequestURL) pair, either choose a backend (by
> name or IP) to handle the request *or* issue an arbitrary redirect
> *or* return an HTTP error.
>=20
> According to the "squid config translation" docs, the equivalent for
> url_rewrite_program is remap.config.  Since that does not actually
> support external programs, it appears the intent is to use the
> built-in remapping abilities to replicate the functionality of the
> external program.  However, that is not always possible.
>=20
> The messy part that prevents us from doing that or hacking up a module
> is that the external program we use with squid is ridiculously
> complicated, pulling together information from multiple sources, all
> of them dynamic, and gets linked to all kinds of libraries like MySQL,
> memcached, and distributed message passing stuff.
>=20
> What is the best way to migrate this functionality to ATS?  The whole
> big ball of wax seems like a really poor candidate for a built-in ATS
> module due to all the external dependencies, many of which run
> asynchronously and can block.  It seems really advantageous to seal
> that off So what we probably really need is a pretty efficient way to
> emulate squid's ability to call out to an external program.


Dealing with external squid helpers is a bit wonky, but you probably =
could implement something in a plugin that does it. The fact that you =
are doing so much weirdness (MySql, Memcached) makes it particularly =
tough, I=E2=80=99m not sure how Squid deals with that?

Dealing with synchronous APIs such as MySQL is tricky to say the least.

I=E2=80=99m not sure we have a good answer here, other than you probably =
need to try to implement this as a plugin. Alternatively, I know some =
people have looked at ICAP (http://www.rfc-editor.org/rfc/rfc3507.txt), =
but I don=E2=80=99t know if that helps you at all either. And, we still =
don=E2=80=99t have that in our code, so whoever has ICAP implemented, =
please open source it :-).

=E2=80=94 Leif