Return-Path: X-Original-To: apmail-trafficserver-users-archive@www.apache.org Delivered-To: apmail-trafficserver-users-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4EEE0104FB for ; Fri, 27 Feb 2015 09:17:26 +0000 (UTC) Received: (qmail 82423 invoked by uid 500); 27 Feb 2015 09:17:19 -0000 Delivered-To: apmail-trafficserver-users-archive@trafficserver.apache.org Received: (qmail 82359 invoked by uid 500); 27 Feb 2015 09:17:19 -0000 Mailing-List: contact users-help@trafficserver.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: users@trafficserver.apache.org Delivered-To: mailing list users@trafficserver.apache.org Received: (qmail 82349 invoked by uid 99); 27 Feb 2015 09:17:19 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 27 Feb 2015 09:17:19 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of degreane@gmail.com designates 209.85.217.180 as permitted sender) Received: from [209.85.217.180] (HELO mail-lb0-f180.google.com) (209.85.217.180) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 27 Feb 2015 09:17:14 +0000 Received: by lbvn10 with SMTP id n10so16028861lbv.4 for ; Fri, 27 Feb 2015 01:16:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=TZ7slkNEwZPFPJgj8hb80pYlN3zYdiWZb1b8I0+JgIU=; b=YrBxUxXtBvzoKL/YR/Ej6Zwu0A9SXZmpaizfaUhrLSA/B0+94F2g467qFjxTL85Loz 1589Oo45ii392QQUNEckdY3Nm57KM2lgMhkqoSXxa1t4rAtQnIxktgCiyO96UmweJKdU wbV5qmW9OZ9pYDe/eYoXPppFN87VhV0KIm5u0JRa8d4rjfqD79xtEIOPDTVfdiFZKHGc qtMaZLUblPO4474HjvwCItSo5fRXIaiTtx8NzXJxLoHvFUxRCp1gOI+y1n5dEPQ4nSBs QvXsCkgWCM7CiyPyqY1icxPso9Doxl0JecsVzCZLPe8jknTP8vV0qnFMaO/zLAnwi82u 2eiA== MIME-Version: 1.0 X-Received: by 10.112.12.134 with SMTP id y6mr11739511lbb.34.1425028613153; Fri, 27 Feb 2015 01:16:53 -0800 (PST) Received: by 10.112.222.139 with HTTP; Fri, 27 Feb 2015 01:16:53 -0800 (PST) In-Reply-To: References: Date: Fri, 27 Feb 2015 11:16:53 +0200 Message-ID: Subject: Re: Migrating from squid From: Faysal Banna To: users@trafficserver.apache.org, j.david.lists@gmail.com Content-Type: multipart/alternative; boundary=001a11c3a0e8482a3705100e5444 X-Virus-Checked: Checked by ClamAV on apache.org --001a11c3a0e8482a3705100e5444 Content-Type: text/plain; charset=UTF-8 Sir. the best way to use or migrate that feature is to do ts_lua .. i been doing it for a while now and it works perfectly it has everything one needs and you can at any time use os.execute from inside Lua script or you can use local handle = io.popen(command)local result = handle:read("*a") handle:close() if you need a result. and its asynchronous with the system so it won't block other operations in ATS. its fast and pretty good to use. i use it lately to rate-limit incoming connections from origin servers and save bandwidth for certain object downloads. if you need any assistance i may be able to help much regards On Fri, Feb 27, 2015 at 7:51 AM, J David wrote: > (Not sure if this is best for the user list or the dev list, as it's > development, but not of ATS per se.) > > Squid offers a feature called url_rewrite_program that can be used to > change its behavior based on client IP, requested URL, and various > other criteria. It runs a bunch of copies of an external program and > passes information to and from them over pipes. > > Overall, ATS is a much better fit for our environment, but we are > heavily dependent on this feature of Squid. > > The short version of what we need to do is this: > > Based on a (ClientIP,RequestURL) pair, either choose a backend (by > name or IP) to handle the request *or* issue an arbitrary redirect > *or* return an HTTP error. > > According to the "squid config translation" docs, the equivalent for > url_rewrite_program is remap.config. Since that does not actually > support external programs, it appears the intent is to use the > built-in remapping abilities to replicate the functionality of the > external program. However, that is not always possible. > > The messy part that prevents us from doing that or hacking up a module > is that the external program we use with squid is ridiculously > complicated, pulling together information from multiple sources, all > of them dynamic, and gets linked to all kinds of libraries like MySQL, > memcached, and distributed message passing stuff. > > What is the best way to migrate this functionality to ATS? The whole > big ball of wax seems like a really poor candidate for a built-in ATS > module due to all the external dependencies, many of which run > asynchronously and can block. It seems really advantageous to seal > that off So what we probably really need is a pretty efficient way to > emulate squid's ability to call out to an external program. > > If that's true, has anyone done something similar? Which ATS module > hook(s) would be the best to use, and how would we handle the need to > yield while we wait for the external program to work? > > Thanks for any advice! > -- ============================ Faysal Banna Meteorological Services Rafic Harriri International Airport Beirut - Lebanon Mob: +961-3-258043 ============================= --001a11c3a0e8482a3705100e5444 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Sir.
the best way to use or migrate that feature is to= do ts_lua .. i been doing it for a while now and it works perfectly=C2=A0<= /div>

it has everything one needs and you can at any tim= e use=C2=A0os.execute fro= m inside Lua script=C2=A0
or you can use=C2=A0
local handle =3D io.po=
pen(command)
local result =3D handle=
:read("*a")
handle:close()
if you need a =
result. and its asynchronous with the system so it won't block other op=
erations in ATS.
its fast and pretty good to use.

i use it lately to rate-limit incoming connectio=
ns from origin servers and save bandwidth for certain object downloads.
if you need any assistance i may be able to help<=
/pre>
much regards

On Fri, Feb 27, 2015 at 7:51 AM, J Davi= d <j.david.lists@gmail.com> wrote:
(Not sure if this is best for the user list or the dev lis= t, as it's
development, but not of ATS per se.)

Squid offers a feature called url_rewrite_program that can be used to
change its behavior based on client IP, requested URL, and various
other criteria.=C2=A0 It runs a bunch of copies of an external program and<= br> passes information to and from them over pipes.

Overall, ATS is a much better fit for our environment, but we are
heavily dependent on this feature of Squid.

The short version of what we need to do is this:

Based on a (ClientIP,RequestURL) pair, either choose a backend (by
name or IP) to handle the request *or* issue an arbitrary redirect
*or* return an HTTP error.

According to the "squid config translation" docs, the equivalent = for
url_rewrite_program is remap.config.=C2=A0 Since that does not actually
support external programs, it appears the intent is to use the
built-in remapping abilities to replicate the functionality of the
external program.=C2=A0 However, that is not always possible.

The messy part that prevents us from doing that or hacking up a module
is that the external program we use with squid is ridiculously
complicated, pulling together information from multiple sources, all
of them dynamic, and gets linked to all kinds of libraries like MySQL,
memcached, and distributed message passing stuff.

What is the best way to migrate this functionality to ATS?=C2=A0 The whole<= br> big ball of wax seems like a really poor candidate for a built-in ATS
module due to all the external dependencies, many of which run
asynchronously and can block.=C2=A0 It seems really advantageous to seal that off So what we probably really need is a pretty efficient way to
emulate squid's ability to call out to an external program.

If that's true, has anyone done something similar?=C2=A0 Which ATS modu= le
hook(s) would be the best to use, and how would we handle the need to
yield while we wait for the external program to work?

Thanks for any advice!



--
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
=C2=A0 =C2=A0 =C2=A0 =C2=A0=C2=A0 Faysal = Banna
=C2=A0Meteorological Services
Rafic Harriri International Airpo= rt
=C2=A0 =C2=A0 =C2=A0 Beirut - Lebanon
=C2=A0 =C2=A0 Mob: +961-3-25= 8043
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D
--001a11c3a0e8482a3705100e5444--