tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From André Warnier (tomcat)>
Subject Re: Strange URL rewrite when reverse proxy with Apache HTTP Server
Date Fri, 24 Feb 2017 00:00:23 GMT
To avoid getting totally confused, you may want to read this explanation first :

In other words, when you send this request from the browser :

The first response that /should/ come back, would be a "re-direct" response from the 
(final) server, with a special HTTP header added :

Then your browser should re-issue the request automatically, this time to :
(with the trailing "/" added)

Then, in a second step, what should happen is :

Your browser indicates a URL, which on the server resolves to a directory.
The server /may/ be configured in such a way, that for this kind of request, it returns an

auto-generated list of all the files in that directory, automatically.
Or it may be configured to return instead, a specific html page.
(That what you do in Apache, with the DirectoryIndex directive)
And if the server is not configured at all for that, it would probably return a response 
"Forbidden", for security reasons (you probably don't want someone to be able to list the

files in that directory, unless you specifically configured the server to allow that).

Or, the server may be configured so that for such a request, it defaults to running some 
standard servlet which is part of the application, and which would return the application

starting page.
That would probably be the case in your "/SelfService" tomcat webapp.

So, the 302 responses which you get in some cases, are probably normal, as explained by 
all the above.
What is definitely /not/ normal, is that port that you get either in these redirect 
responses, or in the error messages.  Because that means that, somewhere in the chain, one

of the elements is not doing its job properly when proxying.
The point of the ProxyPassReverse directive, is precisely that : ensure that if the 
"proxied to" server returns a re-direct response, the "Location:" header that is in that 
redirect response would be properly edited before returning that redirect response back up

the chain, in such a way that the ultimate client would not try next, to access a URL that

leads nowhere.

And now it's really late..

On 24.02.2017 00:06, André Warnier (tomcat) wrote:
> On 23.02.2017 21:17, Aaron Gray wrote:
>> Another weird thing...
>> If you go to:
>> If I am viewing the headers immedaitely I see 302 not found.
> 302 is not "not found". It is "Found", and it is a Redirect response.
> See :
> for a good explanation.
> (But stop after the second paragraph, otherwise you'll get confused again..)
>   Then I wait
>> the 30 sec for it to totally fail and change the URL in the browser to
>> If you just remove the :232700 and hit Enter, it then loads properly.  See
>> immedaitly 200 OK and it is fine.  Don't navigate anywhere else like to
>> or else you'll have to repeat the
>> same thing as above with the :232700 and removing it, hit Enter.
>> Weird.
>> Of course none of this is an issue if you dont hit it with the F5 Load
>> Balancer to start.
> Yes. There seems to be something weird *when a Redirect response is being sent back by
> tomcat*, and has to go back through the load balancer.
> And it may be linked to the fact that the load-balancer does not seem to remove this
> :23270 port which eventually reaches the browser.
>> On Thu, Feb 23, 2017 at 10:24 AM, Aaron Gray <> wrote:
>>> I randomly tried
>>> It works perfectly.  No issues.  I have no idea why its needed to add on
>>> the /index.html (i'm lucky this exists in the tomcat .war, as it was a
>>> guess).  This is visually not appealing, but workable.  Maybe I can do some
>>> url rewrite or something, IF this cant be fixed normally?
>>> On Thu, Feb 23, 2017 at 10:10 AM, Aaron Gray <> wrote:
>>>> Okay, guys, please accept my most sincere apology on this thing.  I just
>>>> figured 1 thing out.
>>>> browser -> f5:443 -> httpd (23270, https) -> tomcat (http)
>>>> using the /static works! (see below)
>>>> If /static/index.html wasnt specified, then it hangs, then comes back as
>>>> But if you go it
>>>> works!!!
>>>> DirectoryIndex index.html
>>>> is present.  because if you just go to https://webserver1:23270/static
>>>> it works fine, no need for the extra /index.html
>>>> not sure why that is different there.
>>>> I only found this out using curl and just happened to think of it.
>>>> So you see the difference, from the tomcat access log, in the two
>>>> variations.
>>>> - - [23/Feb/2017:10:00:48 -0800] "GET /static HTTP/1.1" 302
>>>> -
>>>> - - [23/Feb/2017:10:03:04 -0800] "GET /static/index.html
>>>> HTTP/1.1" 200 32
>>>> Still /SelfService wont work though
>>>> I cant think of a way to do /SelfService/something...
>>>> Since it is a web app and i am no developer and its all from the vendor
>>>> (BMC Software) and it never shows naything other than /SelfService in the
>>>> URL.  I dont think its a frame, but once again i dont know these things.
>>>> know they told me it is HTML5, as it was using Silverlight back in the
>>>> older release.
>>>> On Thu, Feb 23, 2017 at 9:22 AM, Aaron Gray <>
>>>>> Working with my F5 guy, we had an idea, since 80/tcp and 443/tcp were
>>>>> already open to the VIP on the F5, we simply turned of 80 -> 443 redirect
>>>>> on the F5, and then configured the F5 to use the non-HTTP port in Apache
>>>>> HTTP Server (the two backend servers).  So its HTTP the entire way through,
>>>>> and the same issue persisted.
>>>>> browser -> F5(http) -> httpd(http port) -> tomcat(http port,
>>>>> So i think i can rule out any possibility that the fact I was
>>>>> terminating SSL in Apache HTTP Server then proxying clear-text back as
>>>>> reason.  It worked fine like I said if you just go direct to the HTTP
>>>>> Server and slip the F5.   And F5 works landing the /healthcheck.html
>>>>> KeepAlive in the root, which isnt proxied), so all good there.  Just
>>>>> /SelfService (configured w/ Proxy)
>>>>> thanks for your support here though.
>>>>> On Thu, Feb 23, 2017 at 9:12 AM, Aaron Gray <>
>>>>> wrote:
>>>>>> SSLProxyEngine On
>>>>>> Was already turned on this entire time inside the ssl.conf (I include
>>>>>> it) VirtualHost section.
>>>>>> I am debating turning on HTTPS in Tomcat on the backend 10.x.x.x
>>>>>> server, and then HTTPS the whole way through and see if that makes
>>>>>> difference.  I may need to request a new firewall to be opened, which
>>>>>> not able to open until 3/2.  Gonna see.
>>>>>> On Thu, Feb 23, 2017 at 8:22 AM, Antonio S. Cofiño <>
>>>>>> wrote:
>>>>>>> On 23/02/17 12:43, André Warnier (tomcat) wrote:
>>>>>>>> On 22.02.2017 19:22, Aaron Gray wrote:
>>>>>>>>> So this is interesting.
>>>>>>>>> So from HTTP server #1 ( example) I hit:
>>>>>>>>> and I see this in the
HTTP log:
>>>>>>>>> - - [22/Feb/2017:10:14:48 -0800] "GET /static/
>>>>>>>>> 200 32
>>>>>>>>> I see this in the Tomcat log:
>>>>>>>>> - - [22/Feb/2017:10:14:48 -0800] "GET /static/
>>>>>>>>> 200 32
>>>>>>>>> ((( Yes I know they look identical, but the logformat
is the same,
>>>>>>>>> so it
>>>>>>>>> comes out looking the same, but they are  two diff log
files totally
>>>>>>>>> )))
>>>>>>>>> I then hit the
from my Win10
>>>>>>>>> laptop
>>>>>>>>>    ( example)
>>>>>>>>> and I see this in the HTTP server #1 log, but NOTHING
in the tomcat
>>>>>>>>> access
>>>>>>>>> log.
>>>>>>>>> - - [22/Feb/2017:10:20:02 -0800] "-" 408 -
>>>>>>>>> So 408.  Timeout.  Hmm... Why?  We KNOW that it can connect
>>>>>>>>> http ->
>>>>>>>>> tomcat:18080 perfectly.   So is it a timeout BACK to
the F5? That i
>>>>>>>>> dont
>>>>>>>>> know yet.
>>>>>>>> It looks as if, in that case, httpd is trying to connect
to tomcat,
>>>>>>>> but
>>>>>>>> - either it /is/ connecting, and sending the request, but
>>>>>>>> never answers
>>>>>>>> - or it is connecting to something else, which isn't tomcat
>>>>>>>> (but which accepts the request and doesn't answer either)
>>>>>>>> I am not *sure*, but I believe, that if httpd was trying
to connect
>>>>>>>> to something that isn't tomcat, and is not listening on port
18080, it
>>>>>>>> would get a different error, and a different logfile message.
 So it would
>>>>>>>> seem that at least the TCP connection works.
>>>>>>> Is the F5 loadbalancer configured to make reverse proxy to a
SSL http?
>>>>>>> The directive SSLProxyEngine onthe F5 loadbalencer has to be
>>>>>>> If the loadbalancer it's delegating the SSL work to the server1
>>>>>>> server1 https listen =
>>>>>>> then the F5 could be connecting as http regular TCP connection
to a
>>>>>>> SSL TCP one and therefor the SSL handshake it's not happening.
>>>>>>> Antonio
>>>>>>>> Now there is still a puzzle there : when you connect as
>>>>>>>> browser --> httpd --> tomcat
>>>>>>>> then everything works : httpd connects to tomcat, sends a
>>>>>>>> gets a response, and returns it to the browser.
>>>>>>>> But when you do the same as :
>>>>>>>> browser --> F5 --> httpd --> tomcat
>>>>>>>> it "does not work", and the problem seems to be at httpd
--> tomat
>>>>>>>> Why would it make a difference in httpd --> tomcat, if
it is either
>>>>>>>> the browser or the F5 which sends the request to httpd ?
>>>>>>>> the httpd --> tomcat connection should be the same, no
matter what.
>>>>>>>> Hmm, for now as puzzled as you are.
>>>>>>>> Maybe we have a quantum phenomenon here, where the mere fact
of you
>>>>>>>> looking at the logfile, changes what happened beforehand.
Just kidding.
>>>>>>>> Informational note : the Apache access log line, is written
>>>>>>>> Apache *has finished* processing the request (including the
>>>>>>>> round-trip through tomcat) and has sent the response to the
client. (That's
>>>>>>>> because, for example, it can also write the time spent, and
the size of the
>>>>>>>> response; so it has to know those before it can write the
log line).
>>>>>>>> I presume that it is the same for tomcat.
>>>>>>>> Just mentioning this, so that you would maybe have a second
look at
>>>>>>>> the tomcat access log, after a few minutes, to make sure
that there really
>>>>>>>> is no corresponding line there.
>>>>>>>> It could be that tomcat actually /is/ receiving the request,
>>>>>>>> processing it, but that it takes a long time, and that httpd
tires of
>>>>>>>> waiting and already times out in the meantime. In such a
case, you would
>>>>>>>> probably see an error some time later in the tomcat error
log, when it is
>>>>>>>> trying to eventually return the response, and finds a closed
>>>>>>>> instead.
>>>>>>>> Now again, why it would do that in one case and not the other,
>>>>>>>> still a mystery.
>>>>>>>> Another note : in the "timeout" log line above, httpd is
not showing
>>>>>>>> the request URL which triggered this timeout.
>>>>>>>> a) it may be showing some additional information in the Apache
>>>>>>>> log
>>>>>>>> b) these exists an httpd add-on module (mod_log_forensic
?) which
>>>>>>>> could provide more details in the log, about what happens
>>>>>>>>> On Tue, Feb 21, 2017 at 3:39 PM, André Warnier (tomcat)
>>>>>>>>> wrote:
>>>>>>>>> On 21.02.2017 23:28, Aaron Gray wrote:
>>>>>>>>>> Antonio:  The Tomcat server has no knowledge of the
F5, or that it
>>>>>>>>>>> is
>>>>>>>>>>> being
>>>>>>>>>>> fronted by an Apache HTTP Server.  I do SSL termination
in Apache
>>>>>>>>>>> HTTP
>>>>>>>>>>> Server, and clear-text from HTTP to Tomcat.
>>>>>>>>>>> My redirect port for the normal HTTP listen in
Tomcat is commented
>>>>>>>>>>> out.
>>>>>>>>>>>        <Connector port="18080" protocol="HTTP/1.1"
>>>>>>>>>>>                   connectionTimeout="20000" />
>>>>>>>>>>>        <!-- A "Connector" using the shared
thread pool-->
>>>>>>>>>>>        <!--
>>>>>>>>>>>        <Connector executor="tomcatThreadPool"
>>>>>>>>>>>                   port="8080" protocol="HTTP/1.1"
>>>>>>>>>>>                   connectionTimeout="20000"
>>>>>>>>>>>                   redirectPort="8443" />
>>>>>>>>>>>        -->
>>>>>>>>>>> Andre:
>>>>>>>>>>> The URL I am using is
>>>>>>>>>>> It is listening on port 80 and 443, if you hit
80, internally it
>>>>>>>>>>> redirects
>>>>>>>>>>> you to 443.  No SSL cert on the F5 load balancer.
 It simply sends
>>>>>>>>>>> the
>>>>>>>>>>> traffic to one of the two HTTP servers (round-robin,
also tried
>>>>>>>>>>> persistence, no difference).  The HTTP server
is listening only
>>>>>>>>>>> for HTTPS
>>>>>>>>>>> on 23270/tcp.
>>>>>>>>>>> When I access these DMZ webservers via the F5
load balancer (to
>>>>>>>>>>> which I
>>>>>>>>>>> dont have access to, but the network folks configure
for me), it
>>>>>>>>>>> hangs.
>>>>>>>>>>> Eventually returns: server1 https listen =
>>>>>>>>>>> cant load.
>>>>>>>>>>> No idea why the URL is being re-written with
the ":23270".
>>>>>>>>>>> I added static content to the server.xml on
(Tomcat) to
>>>>>>>>>>> test:
>>>>>>>>>>> <Context docBase="/path/to/tomcat/static"
path="/static" />
>>>>>>>>>>> Then put a simple index.html in there.  Accessing
via the Apache
>>>>>>>>>>> Web
>>>>>>>>>>> Servers works fine, but if you hit it with the
Load Balancer it
>>>>>>>>>>> once
>>>>>>>>>>> again
>>>>>>>>>>> adds the
>>>>>>>>>>> Hitting
>>>>>>>>>>> I see my "Hello world!" which is all that is
in index.html. This
>>>>>>>>>>> is the
>>>>>>>>>>> DocumentRoot of HTTP, and *not* proxied over
at this time.
>>>>>>>>>> So in this case, there is no delay, and you get the
>>>>>>>>>> httpd-hosted
>>>>>>>>>> "index.html" containing "Hello World. Right ?
>>>>>>>>>>     Only
>>>>>>>>>> /SelfService and /static are proxied
>>>>>>>>>>> /static just being my test of static content,
but still served up
>>>>>>>>>>> by
>>>>>>>>>>> Tomcat..
>>>>>>>>>>> It's exactly 30 seconds before the page cannot
be loaded when
>>>>>>>>>>> trying
>>>>>>>>>>> anything proxied to Tomcat, but also accessed
via the F5 load
>>>>>>>>>>> balancer.
>>>>>>>>>>> Not sure where the 30 seconds comes from; perhaps
a load balancer
>>>>>>>>>>> time
>>>>>>>>>>> out,
>>>>>>>>>>> as I dont see a "30" in my httpd configurations
or my tomcat
>>>>>>>>>>> server.xml
>>>>>>>>>>> You can certainly look at the Apache httpd logs,
and the tomcat
>>>>>>>>>> logs, to
>>>>>>>>>> see if you get a request or not.
>>>>>>>>>> In Apache httpd, you can set the loglevel individually
>>>>>>>>>> mod_proxy (if
>>>>>>>>>> you are running v 2.4), and it should show something
if it gets this
>>>>>>>>>> request and forwards it to tomcat.
>>>>>>>>>> In tomcat, you can either enable an access log (which
will show if
>>>>>>>>>> it
>>>>>>>>>> receives this request), or you could temporarily
remove/rename the
>>>>>>>>>> /static
>>>>>>>>>> webapp. This way, it should trigger an error "not
found" which you
>>>>>>>>>> would
>>>>>>>>>> also see in the error log.
>>>>>>>>>> There should be nothing between them to hinder it.
 We have many
>>>>>>>>>> load
>>>>>>>>>>> balancers and this one specifically you dont
need to open any
>>>>>>>>>>> firewall
>>>>>>>>>>> requests for the specific networks the HTTP servers
are on. I did
>>>>>>>>>>> have to
>>>>>>>>>>> get the firewall opened up to allow me to hit
>>>>>>>>>>> because the VIP
for "
>>>>>>>>>>> is in the DMZ, and my Desktop & VPN networks
cannot hit it on
>>>>>>>>>>> 80/443
>>>>>>>>>>> without opening holes.  But beyond that, any
connection from the
>>>>>>>>>>> F5 to the
>>>>>>>>>>> HTTP Server should be 100% open bi-directional,
since same subnet..
>>>>>>>>>>> But something isn't working, otherwise you would
not be asking.
>>>>>>>>>> So,
>>>>>>>>>> a) hitting the tomcat webapps through httpd seems
to be working fine
>>>>>>>>>>     (browser -> httpd:23270 -> tomcat:18080
-> webapp or static)
>>>>>>>>>> b) hitting a non-proxied-to-tomcat resource of httpd
seems to work
>>>>>>>>>> fine
>>>>>>>>>> too, even through the F5
>>>>>>>>>>     (browser -> F5:443 -> httpd:23270 ->
html page)
>>>>>>>>>> c) it is only when you do :
>>>>>>>>>>     (browser -> F5:443 -> httpd:23270 ->
tomcat:18080 -> webapp or
>>>>>>>>>> static)
>>>>>>>>>>     that you see this issue
>>>>>>>>>> It would really help if you looked in the logs of
both httpd and
>>>>>>>>>> tomcat,
>>>>>>>>>> and checked for differences betweens cases a, b and
c above.
>>>>>>>>>> I believe that the F5 message with the port 23270
is a minor issue,
>>>>>>>>>> of
>>>>>>>>>> information disclosure by the F5, that it should
not disclose.
>>>>>>>>>> But the reason why it returns this error is obviously
that in that
>>>>>>>>>> case,
>>>>>>>>>> it does not get a response from his request to httpd.
>>>>>>>>>> The reason for this response not coming back to the
F5 (in case c
>>>>>>>>>> only),
>>>>>>>>>> can be due to either httpd or tomcat. But F5 doesn't
know about
>>>>>>>>>> tomcat. So
>>>>>>>>>> for the F5, it is httpd which is not responding.
>>>>>>>>>> - either httpd is never getting the request from
the F5 (unlikely,
>>>>>>>>>> because
>>>>>>>>>> in b above it gets it and responds)
>>>>>>>>>> - or httpd is getting the request from the F5, but
not forwarding
>>>>>>>>>> it to
>>>>>>>>>> tomcat, but also not returning an immediate error
response to F5
>>>>>>>>>> (which
>>>>>>>>>> seems also unlikely, because of a and b)
>>>>>>>>>> - or httpd is getting the request, forwarding it
to tomcat, but not
>>>>>>>>>> getting a response from tomcat. So
>>>>>>>>>>     - either tomcat is never getting the request
from httpd (but in
>>>>>>>>>> a, it
>>>>>>>>>> gets it)
>>>>>>>>>>     - or tomcat is getting the request from httpd,
but not
>>>>>>>>>> responding (but
>>>>>>>>>> in a, it does)
>>>>>>>>>>     - or tomcat is getting the request and responding,
but the
>>>>>>>>>> response
>>>>>>>>>> never gets back to httpd (but in a, it does)
>>>>>>>>>> So if a and b and c are all accurate, there is something
>>>>>>>>>> illogical happening.
>>>>>>>>>> This would lead to the conclusion that a and b and
c cannot all be
>>>>>>>>>> accurate.
>>>>>>>>>> The logs.. ?
>>>>>>>>>> On Tue, Feb 21, 2017 at 2:05 PM, Antonio S. Cofino
>>>>>>>>>>> wrote:
>>>>>>>>>>> Aaron, on tomcat instances change the redirectPort
attributte on
>>>>>>>>>>> the http
>>>>>>>>>>>> conectó to the loabbalancer's port 443
>>>>>>>>>>>> My guess is that your webapp has restriction
rule requesting SSL
>>>>>>>>>>>> con
>>>>>>>>>>>> fidntial channel. Therefore the non-confidential
to the 18080
>>>>>>>>>>>> port from
>>>>>>>>>>>> the
>>>>>>>>>>>> balancer are redirected to the 23270 port,
but it should be 443.
>>>>>>>>>>>> Antonio
>>>>>>>>>>>> El 21/2/2017 19:46, "Aaron Gray" <>
>>>>>>>>>>>> I have an application server from a vendor
that comes bundled
>>>>>>>>>>>> with an
>>>>>>>>>>>> additional Apache Tomcat server.  The webapp
SelfService.war is
>>>>>>>>>>>> vendor
>>>>>>>>>>>> supplied too.
>>>>>>>>>>>> Here's my problem (IP's replaced to protect
the innocent):
>>>>>>>>>>>> networks:
>>>>>>>>>>>> DMZ=172.x.x.x
>>>>>>>>>>>> INTERNAL=10.x.x.x
>>>>>>>>>>>> server1 https listen =
>>>>>>>>>>>> server2 https listen =
>>>>>>>>>>>> F5 load balancer hostname =
>>>>>>>>>>>> backend tomcat server =
>>>>>>>>>>>> mod_proxy configuration:
>>>>>>>>>>>> ProxyPass /SelfService
>>>>>>>>>>>> ProxyPassReverse /SelfService
>>>>>>>>>>>> When I access these DMZ webservers which
mod_proxy back to Apache
>>>>>>>>>>>> Tomcat
>>>>>>>>>>>> as:
>>>>>>>>>>>> and
>>>>>>>>>>>> <
>>>>>>>>>>>>> They load properly. Perfectly, every
>>>>>>>>>>>> When I access these DMZ webservers via the
F5 load balancer (to
>>>>>>>>>>>> which I
>>>>>>>>>>>> dont have access to, but the network folks
configure for me), it
>>>>>>>>>>>> hangs.
>>>>>>>>>>>> Eventually returns:
>>>>>>>>>>>> cant load.
>>>>>>>>>>>> No idea why the URL is being re-written with
the ":23270".
>>>>>>>>>>>> I added static content to the server.xml
on (Tomcat) to
>>>>>>>>>>>> test:
>>>>>>>>>>>> <Context docBase="/path/to/tomcat/static"
path="/static" />
>>>>>>>>>>>> Then put a simple index.html in there.  Accessing
via the Apache
>>>>>>>>>>>> Web
>>>>>>>>>>>> Servers works fine, but if you hit it with
the Load Balancer it
>>>>>>>>>>>> once
>>>>>>>>>>>> again
>>>>>>>>>>>> adds the
>>>>>>>>>>>> Do you have any thoughts?  Thanks so much,
I have been working
>>>>>>>>>>>> with this
>>>>>>>>>>>> for weeks now with no success
>>>>>>>>>> ------------------------------------------------------------
>>>>>>>>>> ---------
>>>>>>>>>> To unsubscribe, e-mail:
>>>>>>>>>> For additional commands, e-mail:
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>> To unsubscribe, e-mail:
>>>>>>>> For additional commands, e-mail:
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail:
>>>>>>> For additional commands, e-mail:
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message