From users-return-263499-archive-asf-public=cust-asf.ponee.io@tomcat.apache.org Fri Jan 5 15:44:13 2018 Return-Path: X-Original-To: archive-asf-public@eu.ponee.io Delivered-To: archive-asf-public@eu.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by mx-eu-01.ponee.io (Postfix) with ESMTP id 2BAF6180647 for ; Fri, 5 Jan 2018 15:44:13 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 1C011160C19; Fri, 5 Jan 2018 14:44:13 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 37618160C14 for ; Fri, 5 Jan 2018 15:44:12 +0100 (CET) Received: (qmail 36520 invoked by uid 500); 5 Jan 2018 14:44:10 -0000 Mailing-List: contact users-help@tomcat.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: "Tomcat Users List" Delivered-To: mailing list users@tomcat.apache.org Received: (qmail 36509 invoked by uid 99); 5 Jan 2018 14:44:10 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 05 Jan 2018 14:44:10 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id DD5991A027D for ; Fri, 5 Jan 2018 14:44:09 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.999 X-Spam-Level: * X-Spam-Status: No, score=1.999 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=eyequestion-nl.20150623.gappssmtp.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id hujaGccH_j1g for ; Fri, 5 Jan 2018 14:44:03 +0000 (UTC) Received: from mail-ua0-f171.google.com (mail-ua0-f171.google.com [209.85.217.171]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 520C95F3B5 for ; Fri, 5 Jan 2018 14:44:03 +0000 (UTC) Received: by mail-ua0-f171.google.com with SMTP id 34so2633207uav.5 for ; Fri, 05 Jan 2018 06:44:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eyequestion-nl.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=Z43kHJ2tyCb7TITwr+ewhhWzLk+vbebp/jHL8Cqw99Y=; b=Rjmz6f6M4zCvOlfEcmgkqbhqLrHHIxHSg9wiajTWralKDcxJYmvMAppWatqCYgETCX T75I3aKTKxqrl6bXY6dzM3SulJDzSFLGmWm1ozOWNSOVaqaRDGUYQmoJ862XIX4TchJw Sr8ocUL1QN4cXR7WIe8TAWm+Xvmp7IDb2BGGEfx2veoCj4L63tcgU3+MEL4/7K/Tczv/ MTjDEsaH9WtCzFjeaGCbrkN6+Q2lKD0QVFaY/3Cm8ToeGDr9CNa+fEuEnJU37Aj2M6UE cwnaCd8mZQviZQKbFeBgiQOLm9F3WxQV0bGFPTMJSEhHLdLM77MfBubbh8a8IJlUT5cx 70wg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=Z43kHJ2tyCb7TITwr+ewhhWzLk+vbebp/jHL8Cqw99Y=; b=TzHBkYriAD6mqF4eDpE1AS2mWnsPguc2xEzxxmHmttISdQ+heMQFTitEAwTNIEPpsj ugTpjfpCf7eM3djCBcqgkYT7ZVJPHlQqfp2SBTqHwFNAaU2E0aMUgmAqQuJloEdcIePv LBMETT8N+O2mDtA4+FikgHY3wAf7Ac4PcXQSL4YeCHgdtU6dIZPFi9q4wqsLLAwkmBXs b9B/t8elxG0gPJeAYdkcl4FXlQmuXdP98Chhp6IQh4pKERXTLMVT6dDjFfG9fkj1Zql8 geapwi2jUZXdlCxJQPg8LEVssCs9W3eK2sgEyh+gfvD0IhoA1lislpad13iSOJiw9vOM JdVQ== X-Gm-Message-State: AKwxytesMoQcSHEWsv823RvnpLM6vt4WbxfwMcsVV90MElsuRBHSgh1V B2b9l5EruXmiXcnvfLYuSeJmQkaNaEDyUW7aVAfi+dLzSAo= X-Google-Smtp-Source: ACJfBouFfcIKcyymCgtkeMXZKvVA1G2rXq9SchiXcTMiB13NEwaR9Pcf274/7LGYicve1mp595f74M2PQzhzp81Dng4= X-Received: by 10.176.22.78 with SMTP id l14mr3062234uae.37.1515163436637; Fri, 05 Jan 2018 06:43:56 -0800 (PST) MIME-Version: 1.0 Received: by 10.31.60.210 with HTTP; Fri, 5 Jan 2018 06:43:56 -0800 (PST) In-Reply-To: <410c5aad-c5dc-8117-8073-6839a513bd87@internetallee.de> References: <00a301d3796d$c9cc9360$5d65ba20$@eyequestion.nl> <019e01d37a39$67cd51f0$3767f5d0$@eyequestion.nl> <088e01d383af$1e9ff030$5bdfd090$@eyequestion.nl> <3a79408b-d877-0be2-b4f9-ec72f6b4dbb3@apache.org> <410c5aad-c5dc-8117-8073-6839a513bd87@internetallee.de> From: Harrie Robins Date: Fri, 5 Jan 2018 15:43:56 +0100 Message-ID: Subject: Re: internalProxies regex To: Tomcat Users List Content-Type: multipart/alternative; boundary="001a1144fad46af0770562087a98" --001a1144fad46af0770562087a98 Content-Type: text/plain; charset="UTF-8" All clear. I apologize, I was in fact not masking the backslashes, I did a wrong copy paste from the pattern I was using in my test I tested the following 2 patterns: ^103\.21\.(2(4[4-7]))\.([0- 9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|^103\.22 \.(2(0[0-3]))\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$ 103\.21\.(2(4[4-7]))\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))|103\.22\.(2(0[0-3]))\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5])) Regards, Harrie On 5 January 2018 at 14:46, Felix Schumacher < felix.schumacher@internetallee.de> wrote: > Am 05.01.2018 um 09:47 schrieb Harrie Robins: > >> Hi Mark, >> >> our tomcat application server are fronted by 1. cloudflare, and 2. amazon >> load balancer. >> In apache there is mod_remote IP and I can simply put in CIDR range: >> https://www.cloudflare.com/ips/ that will swallow all those IP and will >> get >> the correct IP to tomcat. >> >> In Tomcat I need >> https://tomcat.apache.org/tomcat-8.5-doc/api/org/apache/cata >> lina/valves/RemoteIpValve.html >> which does not accept CIDR range however. I wrote a regex to match all the >> addresses and it works, it's matching way to many addresses however so I >> rewrote the pattern. My new pattern is not functioning however, so I >> tested >> then pattern in a small application. >> >> In my test I made a list of all addresses in this range: >> ^103\\.21\\.(2(4[4-7]))\\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2( >> [0-4][0-9]|5[0-5]))$|^103\\.22\\.(2(0[0-3]))\\.([0-9]|[1- >> 9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$ >> > > If you configure the valve through the internalProxies attribute, you are > using 'real' strings and don't need to mask the backslashes as you would > have to do with java strings. > > When you look at the documentation, you will find no double backslashes > there.|||| > > And regarding the usage of the anchors '^' and '$'. They are not needed, > either. Tomcat will use match instead of find and thus they are implicitly > added. > > Regards, > Felix > > || > >> I matched all these addresses and it works. When I set in tomcat however >> it >> does not, I have no understanding why not? >> >> Hope you understand what I am trying to do. >> >> thanks >> >> >> >> >> >> On 2 January 2018 at 19:33, Mark Thomas wrote: >> >> On 02/01/18 09:50, Harrie Robins wrote: >>> >>>> I'm still having problems with matching my pattern. >>>> >>>> Right now I'm feeding the following to internalProxies: >>>> >>>> ^103\\.21\\.(2(4[4-7]))\\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2( >>>> >>> [0-4][0-9]|5[0-5]))$|^103\\.22\\.(2(0[0-3]))\\.([0-9]|[1- >>> 9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$ >>> >>>> I created a list of all involved IP addresses and matched those IP >>>> >>> addresses: >>> >>>> java.util.regex.Matcher / java.util.regex.Pattern, please see >>>> >>> https://pastebin.com/Lija7n9k >>> >>>> All addresses from the list I created are matching, just not in tomcat. >>>> >>> What is the value of the remote IP address that is failing to match? You >>> might want to look at writing a short custom Valve to log that and >>> insert it into the Pipeline ahead of the RemoteIpValve. >>> >>> Another option would be to simply remove the RemoteIpValve and write a >>> simple servlet that logs the remote IP. >>> >>> Mark >>> >>> Regards, >>>> >>>> Harrie >>>> >>>> -----Oorspronkelijk bericht----- >>>> Van: Harrie Robins [mailto:harrie@eyequestion.nl] >>>> Verzonden: 21 December 2017 09:55 >>>> Aan: 'Tomcat Users List' >>>> Onderwerp: RE: internalProxies regex >>>> >>>> This makes perfect sense. >>>> I tested my regex, just against wrong engine. >>>> >>>> Thanks for pointing me in the right direction >>>> >>>> -----Oorspronkelijk bericht----- >>>> Van: Konstantin Kolinko [mailto:knst.kolinko@gmail.com] >>>> Verzonden: 20 December 2017 15:19 >>>> Aan: Tomcat Users List >>>> Onderwerp: Re: internalProxies regex >>>> >>>> 2017-12-20 11:37 GMT+03:00 Harrie Robins : >>>> >>>>> Hello everyone, >>>>> >>>>> >>>>> >>>>> I have a question about the remoteipvalve in tomcat 8.5: >>>>> https://tomcat.apache.org/tomcat-8.5-doc/api/org/apache/catalina/valve >>>>> s/Remo >>>>> teIpValve.html >>>>> >>>>> >>>>> >>>>> >>>>> internalProxies >>>>> >>>>> Regular expression that matches the IP addresses of internal proxies. >>>>> If they appear in the remoteIpHeader value, they will be trusted and >>>>> will not appear in the proxiesHeader value >>>>> >>>>> RemoteIPInternalProxy >>>>> >>>>> Regular expression (in the syntax supported by java.util.regex) >>>>> >>>>> 10\.\d{1,3}\.\d{1,3}\.\d{1,3}|192\.168\.\d{1,3}\.\d{1,3}| >>>>> 169\.254\.\d{1,3}\.\d{1,3}|127\.\d{1,3}\.\d{1,3}\.\d{1,3}| >>>>> 172\.1[6-9]{1}\.\d{1,3}\.\d{1,3}|172\.2[0-9]{1}\.\d{1,3}\.\d{1,3}| >>>>> 172\.3[0-1]{1}\.\d{1,3}\.\d{1,3} >>>>> By default, 10/8, 192.168/16, 169.254/16, 127/8 and 172.16/12 are >>>>> >>>> allowed. >>> >>>> >>>>> >>>>> I need to convert some CIDR ranges to regex: >>>>> >>>>> >>>>> my concern is that /d{1,3} wil match too many (non exist) addresses >>>>> >>>>> 103\.21\.24\d[4-7]\.\d[0-9]\d{1,3}|103\.22\.20\d[0-3]\.\d[0-9]\d{1,3}| >>>>> 103\.3 >>>>> 1\.\d[4-7]\.\d[0-9]\d{1,3} >>>>> >>>>> >>>>> >>>>> So I re-wrote using capture groups, below does not function however, >>>>> and I assume it is due to OR (|) which tomcat will affectively see as a >>>>> >>>> new entry? >>> >>>> So I tried escaping, but I cannot get it to work: >>>>> >>>>> 103\.21\.(2(4[4-7]))\.([0-9]\|[1-9][0-9]\|1([0-9][0-9])\|2([0-4][0-9]\ >>>>> |5[0-5 >>>>> ]))|103\.22\.(2(0[0-3]))\.([0-9]\|[1-9][0-9]\|1([0-9][0-9])\|2([0-4][0 >>>>> -9]\|5 >>>>> [0-5])) >>>>> >>>> Your assumption that "tomcat will affectively see as a new entry" is >>>> >>> wrong. >>> >>>> The string is used as whole to initialize a java.util.regex.Pattern(). >>>> Tomcat does not split it. >>>> >>>> You may write a simple program / junit test to test how >>>> java.util.regex.Pattern() processes your value. Or you may run Tomcat >>>> >>> with debugger, >>> >>>> https://wiki.apache.org/tomcat/FAQ/Developing#Debugging >>>> https://wiki.apache.org/tomcat/FAQ/Troubleshooting_ >>>> >>> and_Diagnostics#Common_Troubleshooting_Scenario >>> >>>> AFAIK, '\|' in a regular expression will be interpreted as expecting >>>> >>> literal '|' character in the matched string. No IP address has this >>> character so none will match. >>> >>>> >>>> >>>> Best regards, >>>> Konstantin Kolinko >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org >>>> For additional commands, e-mail: users-help@tomcat.apache.org >>>> >>>> >>>> >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org >>>> For additional commands, e-mail: users-help@tomcat.apache.org >>>> >>>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org >>> For additional commands, e-mail: users-help@tomcat.apache.org >>> >>> >>> > --001a1144fad46af0770562087a98--