Return-Path: X-Original-To: apmail-trafficserver-users-archive@www.apache.org Delivered-To: apmail-trafficserver-users-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 98D1317392 for ; Fri, 9 Jan 2015 21:38:39 +0000 (UTC) Received: (qmail 25949 invoked by uid 500); 9 Jan 2015 21:38:40 -0000 Delivered-To: apmail-trafficserver-users-archive@trafficserver.apache.org Received: (qmail 25887 invoked by uid 500); 9 Jan 2015 21:38:40 -0000 Mailing-List: contact users-help@trafficserver.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: users@trafficserver.apache.org Delivered-To: mailing list users@trafficserver.apache.org Received: (qmail 25871 invoked by uid 99); 9 Jan 2015 21:38:40 -0000 Received: from mail-relay.apache.org (HELO mail-relay.apache.org) (140.211.11.15) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 09 Jan 2015 21:38:40 +0000 Received: from [192.168.201.3] (unknown [73.181.14.238]) by mail-relay.apache.org (ASF Mail Server at mail-relay.apache.org) with ESMTPSA id 01B091A012A; Fri, 9 Jan 2015 21:38:39 +0000 (UTC) Content-Type: multipart/alternative; boundary="Apple-Mail=_894702A1-611D-4479-B6DF-553A2073A428" Mime-Version: 1.0 (Mac OS X Mail 8.1 \(1993\)) Subject: Re: forward proxy - Restricting domains. From: Leif Hedstrom In-Reply-To: Date: Fri, 9 Jan 2015 14:38:36 -0700 Cc: Sudheer Vinukonda Message-Id: <020F6313-BC97-4C0B-8A15-F525BF215F64@apache.org> References: <2116454793.1211049.1420836439863.JavaMail.yahoo@jws100130.mail.ne1.yahoo.com> To: users@trafficserver.apache.org X-Mailer: Apple Mail (2.1993) --Apple-Mail=_894702A1-611D-4479-B6DF-553A2073A428 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Jan 9, 2015, at 2:29 PM, Paul Tader wrote: >=20 > Doesn=E2=80=99t this break the forward proxy then? >=20 > # To enable forward proxy, you must turn off remap_required > CONFIG proxy.config.url_remap.remap_required INT 1 That=E2=80=99s somewhat confusing. remap_required disables =E2=80=9Copen = forward proxying=E2=80=9D. ATS actually doesn=E2=80=99t know / care = about forward vs reverse proxy, it=E2=80=99s just a matter of what = requests you allow through. What this setting is saying =E2=80=9CWithout = an explicit rule matching in remap.config, deny the request=E2=80=9D. = There=E2=80=99s a similar one for reverse proxy. =E2=80=94 Leif >=20 >=20 >> On Jan 9, 2015, at 2:47 PM, Sudheer Vinukonda > wrote: >>=20 >> You will also need to enable the config = proxy.config.url_remap.remap_required (like Leif suggested earlier). >>=20 >>=20 >>=20 >> On Friday, January 9, 2015 12:30 PM, Paul Tader = > wrote: >>=20 >>=20 >> I think this would work, and I think I=E2=80=99m close but I tried = this (ver 3 uses .useflt and .defflt instead of .activatefilter and = .deactivatefilter): >>=20 >>=20 >> .defflt disable_all @action=3Ddeny >> .defflt internal_only @action=3Dallow = @src_ip=3D10.0.0.0-255.255.255.255 >>=20 >> .useflt internal_only >> map https://www.facebook.com = https://www.facebook.com >> map https://www.yahoo.com = https://www.yahoo.com >> map http://finance.yahoo.com = http://finance.yahoo.com >> .unuseflt internal_only >>=20 >> .useflt disable_all >>=20 >>=20 >> But going to a site not listed (www.oracle.com = ) is still allowed. ? >> 1420835169.093 134 10.1.2.3 TCP_MISS/200 38458 GET = http://www.oracle.com/index.html - = DIRECT/www.oracle.com text/html - >>=20 >> I=E2=80=99ve also tried placing ".useflt disable_all=E2=80=9D before = the =E2=80=9C.useflt internal_only=E2=80=9D filter with no luck, sites = not on the list are still allowed out. >>=20 >>=20 >>> On Jan 9, 2015, at 12:02 PM, Sudheer Vinukonda = > wrote: >>>=20 >>> I think you would need to use named_filters to specify ranges in = remap.config. >>>=20 >>>=20 >>> remap.config =E2=80=94 Apache Traffic Server 5.3.0 documentation = >>> =20 >>> =20 >>> =20 >>> =20 >>> =20 >>> =20 >>> remap.config =E2=80=94 Apache Traffic Server 5.3.0 documentation >>> = remap.config The remap.config file (by = default, located in /opt/trafficserver/etc/trafficserver/) contains = mapping rules that Traffic Server uses to perform the following actions: >>> View on docs.trafficserver.apache.org = =09 >>> Preview by Yahoo >>> =20 >>> =20 >>>=20 >>>=20 >>> On Friday, January 9, 2015 9:50 AM, Paul Tader = > wrote: >>>=20 >>>=20 >>>=20 >>>> On Jan 9, 2015, at 10:33 AM, Paul Tader > wrote: >>>>=20 >>>>>=20 >>>>> On Jan 9, 2015, at 10:22 AM, James Peach > wrote: >>>>>=20 >>>>>=20 >>>>>> On Jan 9, 2015, at 8:00 AM, Paul Tader > wrote: >>>>>>=20 >>>>>> Hmm, I didn=E2=80=99t think about a DNS blackhole. For now I=E2=80= =99m looking into additional remap files using the =E2=80=9C.include=E2=80= =9D directive in remap.config but I get these errors after running = traffic_line -x >>>>>>=20 >>>>>> [Jan 9 15:57:04.270] Server {47752783210240} WARNING: Could not = add rule at line #126; Aborting! >>>>>> [Jan 9 15:57:04.270] Server {47752783210240} WARNING: = [ReverseProxy] Unknown directive ".include" at line 126 >>>>>> [Jan 9 15:57:04.270] Server {47752783210240} WARNING: something = failed during BuildTable() -- check your remap plugins! >>>>>> [Jan 9 15:57:04.270] Server {47752783210240} WARNING: failed to = reload remap.config, not replacing! >>>>>>=20 >>>>>> My remap.conf has these two lines: >>>>>>=20 >>>>>> .include /etc/trafficserver/filters.config >>>>>> .include /etc/trafficserver/set1.remap.config >>>>>>=20 >>>>>> =E2=80=A6which is odd because the documentation states: >>>>>>=20 >>>>>> "The .include directive allows mapping rules to be spread across = multiple files. The argument to the .include directive is a list of file = names to be parsed for additional mapping rules. " >>>>>>=20 >>>>>> = http://trafficserver.readthedocs.org/en/latest/reference/configuration/rem= ap.config.en.html = >>>>>=20 >>>>> Does your version of ATS match the version of the docs? >>>>=20 >>>>=20 >>>> Nope and I apologize for that. Time to upgrade. >>>>=20 >>>> Thanks everyone. >>>>=20 >>>=20 >>> Before I upgrade, I=E2=80=99ve tried a =E2=80=9Cdeny all=E2=80=9D = map as the last line in remap.conf and listing all the allowed sites = before this deny line, but it doesn=E2=80=99t take. Can something like = this be done? (ATS version 3.04) >>>=20 >>> ... >>> map http://apache.org/ http://apache = .org @action=3Dallow @src_ip=3D12.34.56.123 >>> map / http://127.0.0.1 = @action=3Ddeny @src_ip=3D0.0.0.1-254.254.254.254 >>>=20 >>>=20 >>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>>> On Jan 8, 2015, at 8:56 PM, Leif Hedstrom > wrote: >>>>>>>=20 >>>>>>>=20 >>>>>>>> On Jan 8, 2015, at 10:53 AM, Paul Tader > wrote: >>>>>>>>=20 >>>>>>>> We have a forward only proxy server configured. How can I = restrict a internal IP address or IP address range to only be able to = proxy certain top level domains (ie google.com , = yahoo.com , etc)? I=E2=80=99ve read a lot on = remapping, but I don=E2=80=99t think that is the correct approach. >>>>>>>=20 >>>>>>>=20 >>>>>>> DNS blackholing as suggested seems like a reasonable solution. = If your list of domains is smallish, then something in remap.config = might work as well. I=E2=80=99ve done this in the past, blocking all but = a few HTTPS sites (via setting remap.required to 1 in records.config). = The other option is to allow all sites, but list the ones that you = intend to block (map them to some nonexistent domain or IP, e.g. = 10.0.0.0). >>>>>>>=20 >>>>>>> Fwiw, remap rules like this with CONNECT methods only works in = 5.0.0 and later. >>>>>>>=20 >>>>>>> =E2=80=94 Leif >>>=20 >>>=20 >>>=20 >>=20 >>=20 >>=20 >=20 --Apple-Mail=_894702A1-611D-4479-B6DF-553A2073A428 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8
On Jan 9, 2015, at 2:29 PM, Paul Tader <ptader@collectivei.com> wrote:

Doesn=E2=80=99t = this break the forward proxy then?

   # To = enable forward proxy, you must turn off remap_required
CONFIG proxy.config.url_remap.remap_required INT = 1

That=E2=80= =99s somewhat confusing. remap_required disables =E2=80=9Copen forward = proxying=E2=80=9D. ATS actually doesn=E2=80=99t know / care about = forward vs reverse proxy, it=E2=80=99s just a matter of what requests = you allow through. What this setting is saying =E2=80=9CWithout an = explicit rule matching in remap.config, deny the request=E2=80=9D. = There=E2=80=99s a similar one for reverse proxy.

=E2=80=94 Leif



On Jan = 9, 2015, at 2:47 PM, Sudheer Vinukonda <sudheerv@yahoo-inc.com> wrote:

You will also need to = enable the config proxy.config.url_remap.remap_required (like Leif = suggested earlier).



=
On Friday, = January 9, 2015 12:30 PM, Paul Tader <ptader@collectivei.com> wrote:
 =


I think this would = work, and I think I=E2=80=99m close but I tried this (ver 3 = uses .useflt and .defflt instead of .activatefilter and = .deactivatefilter):


.defflt  disable_all @action=3Ddeny
.defflt  internal_only @action=3Dallow =  @src_ip=3D10.0.0.0-255.255.255.255

.useflt internal_only
map https://www.facebook.com    https://www.facebook.com
map https://www.yahoo.com       = https://www.yahoo.com
map http://finance.yahoo.com    http://finance.yahoo.com
.unuseflt internal_only

.useflt = disable_all


But going to = a site not listed (www.oracle.com) is still allowed. =  ?
1420835169.093 134 10.1.2.3 = TCP_MISS/200 38458 GET http://www.oracle.com/index.html - DIRECT/www.oracle.com text/html -

I=E2=80=99ve also tried placing ".useflt = disable_all=E2=80=9D before the =E2=80=9C.useflt internal_only=E2=80=9D = filter with no luck, sites not on the list are still allowed = out.


On Jan 9, 2015, at = 12:02 PM, Sudheer Vinukonda <sudheerv@yahoo-inc.com> wrote:

I think you would need to use named_filters to = specify ranges in remap.config.


 


On Friday, January 9, 2015 9:50 AM, Paul Tader = <ptader@collectivei.com> wrote:



On Jan 9, 2015, at = 10:33 AM, Paul Tader <ptader@collectivei.com> wrote:


On Jan 9, 2015, at = 10:22 AM, James Peach <jpeach@apache.org> wrote:


On Jan 9, 2015, at 8:00 AM, Paul Tader <ptader@collectivei.com> wrote:

Hmm, I didn=E2=80=99t = think about a DNS blackhole.  For now I=E2=80=99m looking into = additional remap files using the =E2=80=9C.include=E2=80=9D directive in = remap.config but I get these errors after running traffic_line -x

[Jan  9 = 15:57:04.270] Server {47752783210240} WARNING: Could not add rule at = line #126; Aborting!
[Jan  9 = 15:57:04.270] Server {47752783210240} WARNING: [ReverseProxy] Unknown = directive ".include" at line 126
[Jan  9 = 15:57:04.270] Server {47752783210240} WARNING: something failed during = BuildTable() -- check your remap plugins!
[Jan =  9 15:57:04.270] Server {47752783210240} WARNING: failed to reload = remap.config, not replacing!

My remap.conf has these two lines:

.include = /etc/trafficserver/filters.config
.include = /etc/trafficserver/set1.remap.config

=E2=80=A6which is odd because the documentation = states:

"The = .include directive allows mapping rules to be spread across multiple = files. The argument to the .include directive is a list of file names to = be parsed for additional mapping rules. "

http://trafficserver.readthedocs.org/en/latest/ref= erence/configuration/remap.config.en.html

Does = your version of ATS match the version of the docs?


Nope and I apologize for = that. Time to upgrade.

Thanks everyone.


Before I upgrade, I=E2=80=99ve tried a =E2=80=9Cde= ny all=E2=80=9D map as the last line in remap.conf and listing all the = allowed sites before this deny line, but it doesn=E2=80=99t take. =  Can something like this be done?  (ATS version = 3.04)

...
map http://apache.org/   http://apache.org   @action=3Dallow =   @src_ip=3D12.34.56.123
map / =                    http://127.0.0.1    @action=3Ddeny = @src_ip=3D0.0.0.1-254.254.254.254







On Jan 8, 2015, at 8:56 PM, Leif Hedstrom <zwoop@apache.org> wrote:


On Jan 8, 2015, at 10:53 AM, Paul Tader <ptader@collectivei.com> wrote:

We have a forward = only proxy server configured. How can I restrict a internal IP address = or IP address range to only be able to proxy certain top level domains = (ie google.com, yahoo.com, etc)?  I=E2=80=99ve read a = lot on remapping, but I don=E2=80=99t think that is the correct = approach.


DNS blackholing as = suggested seems like a reasonable solution. If your list of domains is = smallish, then something in remap.config might work as well. I=E2=80=99ve = done this in the past, blocking all but a few HTTPS sites (via setting = remap.required to 1 in records.config). The other option is to allow all = sites, but list the ones that you intend to block (map them to some = nonexistent domain or IP, e.g. 10.0.0.0).

Fwiw, remap rules like this with CONNECT methods = only works in 5.0.0 and later.

=E2=80=94 = Leif



=



=


= --Apple-Mail=_894702A1-611D-4479-B6DF-553A2073A428--