tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron Gray <aaronmg...@gmail.com>
Subject Re: Strange URL rewrite when reverse proxy with Apache HTTP Server
Date Thu, 23 Feb 2017 18:10:28 GMT
Okay, guys, please accept my most sincere apology on this thing.  I just
figured 1 thing out.

browser -> f5:443 -> httpd (23270, https) -> tomcat (http)
using the /static works! (see below)

If /static/index.html wasnt specified, then it hangs, then comes back as
https://loadbalancer.domain.com:23270/static
But if you go https://loadbalancer.domain.com/static/index.html it works!!!

DirectoryIndex index.html
is present.  because if you just go to https://webserver1:23270/static
it works fine, no need for the extra /index.html

not sure why that is different there.

I only found this out using curl and just happened to think of it.
So you see the difference, from the tomcat access log, in the two
variations.

172.18.160.66 - - [23/Feb/2017:10:00:48 -0800] "GET /static HTTP/1.1" 302 -
172.18.160.66 - - [23/Feb/2017:10:03:04 -0800] "GET /static/index.html
HTTP/1.1" 200 32

Still /SelfService wont work though
I cant think of a way to do /SelfService/something...
Since it is a web app and i am no developer and its all from the vendor
(BMC Software) and it never shows naything other than /SelfService in the
URL.  I dont think its a frame, but once again i dont know these things.  I
know they told me it is HTML5, as it was using Silverlight back in the
older release.


On Thu, Feb 23, 2017 at 9:22 AM, Aaron Gray <aaronmgray@gmail.com> wrote:

> Working with my F5 guy, we had an idea, since 80/tcp and 443/tcp were
> already open to the VIP on the F5, we simply turned of 80 -> 443 redirect
> on the F5, and then configured the F5 to use the non-HTTP port in Apache
> HTTP Server (the two backend servers).  So its HTTP the entire way through,
> and the same issue persisted.
>
> browser -> F5(http) -> httpd(http port) -> tomcat(http port, nonssl)
>
> So i think i can rule out any possibility that the fact I was terminating
> SSL in Apache HTTP Server then proxying clear-text back as a reason.  It
> worked fine like I said if you just go direct to the HTTP Server and slip
> the F5.   And F5 works landing the /healthcheck.html (HTTP KeepAlive in the
> root, which isnt proxied), so all good there.  Just cant /SelfService
> (configured w/ Proxy)
>
> thanks for your support here though.
>
> On Thu, Feb 23, 2017 at 9:12 AM, Aaron Gray <aaronmgray@gmail.com> wrote:
>
>> SSLProxyEngine On
>> Was already turned on this entire time inside the ssl.conf (I include it)
>> VirtualHost section.
>>
>> I am debating turning on HTTPS in Tomcat on the backend 10.x.x.x app
>> server, and then HTTPS the whole way through and see if that makes any
>> difference.  I may need to request a new firewall to be opened, which is
>> not able to open until 3/2.  Gonna see.
>>
>>
>> On Thu, Feb 23, 2017 at 8:22 AM, Antonio S. Cofiño <cofinoa@gmail.com>
>> wrote:
>>
>>>
>>> On 23/02/17 12:43, André Warnier (tomcat) wrote:
>>>
>>>> On 22.02.2017 19:22, Aaron Gray wrote:
>>>>
>>>>> So this is interesting.
>>>>>
>>>>> So from HTTP server #1 (172.1.1.1 example) I hit:
>>>>> https://172.1.1.1:23270/static and I see this in the HTTP log:
>>>>> 172.1.1.1 - - [22/Feb/2017:10:14:48 -0800] "GET /static/ HTTP/1.1" 200
>>>>> 32
>>>>> I see this in the Tomcat log:
>>>>> 172.1.1.1 - - [22/Feb/2017:10:14:48 -0800] "GET /static/ HTTP/1.1" 200
>>>>> 32
>>>>> ((( Yes I know they look identical, but the logformat is the same, so
>>>>> it
>>>>> comes out looking the same, but they are  two diff log files totally
>>>>> )))
>>>>>
>>>>> I then hit the https://loadbalancer.domain.com/static from my Win10
>>>>> laptop
>>>>>   (10.1.1.2 example)
>>>>> and I see this in the HTTP server #1 log, but NOTHING in the tomcat
>>>>> access
>>>>> log.
>>>>> 10.1.1.2 - - [22/Feb/2017:10:20:02 -0800] "-" 408 -
>>>>>
>>>>> So 408.  Timeout.  Hmm... Why?  We KNOW that it can connect from http
>>>>> ->
>>>>> tomcat:18080 perfectly.   So is it a timeout BACK to the F5? That i
>>>>> dont
>>>>> know yet.
>>>>>
>>>>
>>>> It looks as if, in that case, httpd is trying to connect to tomcat, but
>>>> - either it /is/ connecting, and sending the request, but tomcat never
>>>> answers
>>>> - or it is connecting to something else, which isn't tomcat
>>>> (but which accepts the request and doesn't answer either)
>>>>
>>>> I am not *sure*, but I believe, that if httpd was trying to connect to
>>>> something that isn't tomcat, and is not listening on port 18080, it would
>>>> get a different error, and a different logfile message.  So it would seem
>>>> that at least the TCP connection works.
>>>>
>>>
>>>
>>> Is the F5 loadbalancer configured to make reverse proxy to a SSL http?
>>> The directive SSLProxyEngine onthe F5 loadbalencer has to be enabled.
>>> http://httpd.apache.org/docs/current/mod/mod_ssl.html#sslproxyengine
>>>
>>> If the loadbalancer it's delegating the SSL work to the server1
>>> server1 https listen = 172.1.1.1:23270
>>>
>>> then the F5 could be connecting as http regular TCP connection to a SSL
>>> TCP one and therefor the SSL handshake it's not happening.
>>>
>>> Antonio
>>>
>>>
>>>
>>>
>>>
>>>
>>>> Now there is still a puzzle there : when you connect as
>>>> browser --> httpd --> tomcat
>>>> then everything works : httpd connects to tomcat, sends a request, gets
>>>> a response, and returns it to the browser.
>>>>
>>>> But when you do the same as :
>>>> browser --> F5 --> httpd --> tomcat
>>>> it "does not work", and the problem seems to be at httpd --> tomat
>>>> Why would it make a difference in httpd --> tomcat, if it is either the
>>>> browser or the F5 which sends the request to httpd ?
>>>> the httpd --> tomcat connection should be the same, no matter what.
>>>>
>>>> Hmm, for now as puzzled as you are.
>>>> Maybe we have a quantum phenomenon here, where the mere fact of you
>>>> looking at the logfile, changes what happened beforehand. Just kidding.
>>>>
>>>> Informational note : the Apache access log line, is written when Apache
>>>> *has finished* processing the request (including the possible round-trip
>>>> through tomcat) and has sent the response to the client. (That's because,
>>>> for example, it can also write the time spent, and the size of the
>>>> response; so it has to know those before it can write the log line).
>>>> I presume that it is the same for tomcat.
>>>> Just mentioning this, so that you would maybe have a second look at the
>>>> tomcat access log, after a few minutes, to make sure that there really is
>>>> no corresponding line there.
>>>> It could be that tomcat actually /is/ receiving the request, and
>>>> processing it, but that it takes a long time, and that httpd tires of
>>>> waiting and already times out in the meantime. In such a case, you would
>>>> probably see an error some time later in the tomcat error log, when it is
>>>> trying to eventually return the response, and finds a closed connection
>>>> instead.
>>>> Now again, why it would do that in one case and not the other, is still
>>>> a mystery.
>>>>
>>>> Another note : in the "timeout" log line above, httpd is not showing
>>>> the request URL which triggered this timeout.
>>>> a) it may be showing some additional information in the Apache error log
>>>> b) these exists an httpd add-on module (mod_log_forensic ?) which could
>>>> provide more details in the log, about what happens precisely
>>>>
>>>>
>>>>
>>>>> On Tue, Feb 21, 2017 at 3:39 PM, André Warnier (tomcat) <aw@ice-sa.com
>>>>> >
>>>>> wrote:
>>>>>
>>>>> On 21.02.2017 23:28, Aaron Gray wrote:
>>>>>>
>>>>>> Antonio:  The Tomcat server has no knowledge of the F5, or that it
is
>>>>>>> being
>>>>>>> fronted by an Apache HTTP Server.  I do SSL termination in Apache
>>>>>>> HTTP
>>>>>>> Server, and clear-text from HTTP to Tomcat.
>>>>>>> My redirect port for the normal HTTP listen in Tomcat is commented
>>>>>>> out.
>>>>>>>       <Connector port="18080" protocol="HTTP/1.1"
>>>>>>>                  connectionTimeout="20000" />
>>>>>>>       <!-- A "Connector" using the shared thread pool-->
>>>>>>>       <!--
>>>>>>>       <Connector executor="tomcatThreadPool"
>>>>>>>                  port="8080" protocol="HTTP/1.1"
>>>>>>>                  connectionTimeout="20000"
>>>>>>>                  redirectPort="8443" />
>>>>>>>       -->
>>>>>>>
>>>>>>> Andre:
>>>>>>> The URL I am using is https://loadbalancer.domain.com
>>>>>>> It is listening on port 80 and 443, if you hit 80, internally
it
>>>>>>> redirects
>>>>>>> you to 443.  No SSL cert on the F5 load balancer.  It simply
sends
>>>>>>> the
>>>>>>> traffic to one of the two HTTP servers (round-robin, also tried
>>>>>>> persistence, no difference).  The HTTP server is listening only
for
>>>>>>> HTTPS
>>>>>>> on 23270/tcp.
>>>>>>> When I access these DMZ webservers via the F5 load balancer (to
>>>>>>> which I
>>>>>>> dont have access to, but the network folks configure for me),
it
>>>>>>> hangs.
>>>>>>> Eventually returns: server1 https listen = 172.1.1.1:23270
>>>>>>>
>>>>>>> https://loadbalancer.domain.com:23270/SelfService
>>>>>>> cant load.
>>>>>>>
>>>>>>> No idea why the URL is being re-written with the ":23270".
>>>>>>> I added static content to the server.xml on 10.1.1.1 (Tomcat)
to
>>>>>>> test:
>>>>>>> <Context docBase="/path/to/tomcat/static" path="/static" />
>>>>>>> Then put a simple index.html in there.  Accessing via the Apache
Web
>>>>>>> Servers works fine, but if you hit it with the Load Balancer
it once
>>>>>>> again
>>>>>>> adds the https://loadbalancer.domain.com:23270/static
>>>>>>>
>>>>>>> Hitting https://loadbalancer.domain.com
>>>>>>> I see my "Hello world!" which is all that is in index.html. This
is
>>>>>>> the
>>>>>>> DocumentRoot of HTTP, and *not* proxied over at this time.
>>>>>>>
>>>>>>>
>>>>>> So in this case, there is no delay, and you get the Apache
>>>>>> httpd-hosted
>>>>>> "index.html" containing "Hello World. Right ?
>>>>>>
>>>>>>    Only
>>>>>>
>>>>>> /SelfService and /static are proxied
>>>>>>> /static just being my test of static content, but still served
up by
>>>>>>> Tomcat..
>>>>>>>
>>>>>>> It's exactly 30 seconds before the page cannot be loaded when
trying
>>>>>>> anything proxied to Tomcat, but also accessed via the F5 load
>>>>>>> balancer.
>>>>>>> Not sure where the 30 seconds comes from; perhaps a load balancer
>>>>>>> time
>>>>>>> out,
>>>>>>> as I dont see a "30" in my httpd configurations or my tomcat
>>>>>>> server.xml
>>>>>>>
>>>>>>>
>>>>>>> You can certainly look at the Apache httpd logs, and the tomcat
>>>>>> logs, to
>>>>>> see if you get a request or not.
>>>>>> In Apache httpd, you can set the loglevel individually for mod_proxy
>>>>>> (if
>>>>>> you are running v 2.4), and it should show something if it gets this
>>>>>> request and forwards it to tomcat.
>>>>>> In tomcat, you can either enable an access log (which will show if
it
>>>>>> receives this request), or you could temporarily remove/rename the
>>>>>> /static
>>>>>> webapp. This way, it should trigger an error "not found" which you
>>>>>> would
>>>>>> also see in the error log.
>>>>>>
>>>>>> There should be nothing between them to hinder it.  We have many
load
>>>>>>
>>>>>>> balancers and this one specifically you dont need to open any
>>>>>>> firewall
>>>>>>> requests for the specific networks the HTTP servers are on. I
did
>>>>>>> have to
>>>>>>> get the firewall opened up to allow me to hit
>>>>>>> https://loadbalancer.domain.com because the VIP for "
>>>>>>> loadbalancer.domain.com"
>>>>>>> is in the DMZ, and my Desktop & VPN networks cannot hit it
on 80/443
>>>>>>> without opening holes.  But beyond that, any connection from
the F5
>>>>>>> to the
>>>>>>> HTTP Server should be 100% open bi-directional, since same subnet.
>>>>>>>
>>>>>>>
>>>>>>> But something isn't working, otherwise you would not be asking.
>>>>>> So,
>>>>>>
>>>>>> a) hitting the tomcat webapps through httpd seems to be working fine
>>>>>>    (browser -> httpd:23270 -> tomcat:18080 -> webapp or
static)
>>>>>>
>>>>>> b) hitting a non-proxied-to-tomcat resource of httpd seems to work
>>>>>> fine
>>>>>> too, even through the F5
>>>>>>    (browser -> F5:443 -> httpd:23270 -> html page)
>>>>>>
>>>>>> c) it is only when you do :
>>>>>>    (browser -> F5:443 -> httpd:23270 -> tomcat:18080 ->
webapp or
>>>>>> static)
>>>>>>    that you see this issue
>>>>>>
>>>>>> It would really help if you looked in the logs of both httpd and
>>>>>> tomcat,
>>>>>> and checked for differences betweens cases a, b and c above.
>>>>>>
>>>>>> I believe that the F5 message with the port 23270 is a minor issue,
of
>>>>>> information disclosure by the F5, that it should not disclose.
>>>>>>
>>>>>> But the reason why it returns this error is obviously that in that
>>>>>> case,
>>>>>> it does not get a response from his request to httpd.
>>>>>> The reason for this response not coming back to the F5 (in case c
>>>>>> only),
>>>>>> can be due to either httpd or tomcat. But F5 doesn't know about
>>>>>> tomcat. So
>>>>>> for the F5, it is httpd which is not responding. Thus,
>>>>>> - either httpd is never getting the request from the F5 (unlikely,
>>>>>> because
>>>>>> in b above it gets it and responds)
>>>>>> - or httpd is getting the request from the F5, but not forwarding
it
>>>>>> to
>>>>>> tomcat, but also not returning an immediate error response to F5
>>>>>> (which
>>>>>> seems also unlikely, because of a and b)
>>>>>> - or httpd is getting the request, forwarding it to tomcat, but not
>>>>>> getting a response from tomcat. So
>>>>>>    - either tomcat is never getting the request from httpd (but in
a,
>>>>>> it
>>>>>> gets it)
>>>>>>    - or tomcat is getting the request from httpd, but not responding
>>>>>> (but
>>>>>> in a, it does)
>>>>>>    - or tomcat is getting the request and responding, but the response
>>>>>> never gets back to httpd (but in a, it does)
>>>>>>
>>>>>> So if a and b and c are all accurate, there is something apparently
>>>>>> illogical happening.
>>>>>> This would lead to the conclusion that a and b and c cannot all be
>>>>>> accurate.
>>>>>>
>>>>>> The logs.. ?
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, Feb 21, 2017 at 2:05 PM, Antonio S. Cofino <cofinoa@gmail.com
>>>>>> >
>>>>>>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Aaron, on tomcat instances change the redirectPort attributte
on the
>>>>>>> http
>>>>>>>
>>>>>>>> conectó to the loabbalancer's port 443
>>>>>>>>
>>>>>>>> My guess is that your webapp has restriction rule requesting
SSL con
>>>>>>>> fidntial channel. Therefore the non-confidential to the 18080
port
>>>>>>>> from
>>>>>>>> the
>>>>>>>> balancer are redirected to the 23270 port, but it should
be 443.
>>>>>>>>
>>>>>>>> Antonio
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> El 21/2/2017 19:46, "Aaron Gray" <aaronmgray@gmail.com>
escribió:
>>>>>>>>
>>>>>>>> I have an application server from a vendor that comes bundled
with
>>>>>>>> an
>>>>>>>> additional Apache Tomcat server.  The webapp SelfService.war
is
>>>>>>>> vendor
>>>>>>>> supplied too.
>>>>>>>>
>>>>>>>> Here's my problem (IP's replaced to protect the innocent):
>>>>>>>>
>>>>>>>> networks:
>>>>>>>> DMZ=172.x.x.x
>>>>>>>> INTERNAL=10.x.x.x
>>>>>>>>
>>>>>>>> server1 https listen = 172.1.1.1:23270
>>>>>>>> server2 https listen = 172.1.1.2:23270
>>>>>>>> F5 load balancer hostname = loadbalancer.domain.com:443
>>>>>>>> backend tomcat server = 10.1.1.1:18080
>>>>>>>>
>>>>>>>> mod_proxy configuration:
>>>>>>>> ProxyPass /SelfService http://10.1.1.1:18080/SelfService
>>>>>>>> ProxyPassReverse /SelfService http://10.1.1.1:18080/SelfService
>>>>>>>>
>>>>>>>> When I access these DMZ webservers which mod_proxy back to
Apache
>>>>>>>> Tomcat
>>>>>>>> as:
>>>>>>>> https://172.1.1.1:23270/SelfService
>>>>>>>> and
>>>>>>>> https://172.1.1.2:23270/SelfService <https://172.1.1.1:23270/SelfS
>>>>>>>> ervice
>>>>>>>>
>>>>>>>>>
>>>>>>>>> They load properly. Perfectly, every time!
>>>>>>>>
>>>>>>>> When I access these DMZ webservers via the F5 load balancer
(to
>>>>>>>> which I
>>>>>>>> dont have access to, but the network folks configure for
me), it
>>>>>>>> hangs.
>>>>>>>> Eventually returns:
>>>>>>>> https://loadbalancer.domain.com:23270/SelfService
>>>>>>>> cant load.
>>>>>>>>
>>>>>>>> No idea why the URL is being re-written with the ":23270".
>>>>>>>> I added static content to the server.xml on 10.1.1.1 (Tomcat)
to
>>>>>>>> test:
>>>>>>>> <Context docBase="/path/to/tomcat/static" path="/static"
/>
>>>>>>>> Then put a simple index.html in there.  Accessing via the
Apache Web
>>>>>>>> Servers works fine, but if you hit it with the Load Balancer
it once
>>>>>>>> again
>>>>>>>> adds the https://loadbalancer.domain.com:23270/static
>>>>>>>>
>>>>>>>> Do you have any thoughts?  Thanks so much, I have been working
with
>>>>>>>> this
>>>>>>>> for weeks now with no success
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>
>>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message