tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Frederik Nosi <frederik.n...@postecom.it>
Subject Re: High thread count & load on Tomcat8 when accessing AJP port with no request
Date Wed, 19 Nov 2014 18:37:29 GMT
Hi Lisa,
On 11/19/2014 07:28 PM, Lisa Woodring wrote:
> On Wed, Nov 19, 2014 at 1:20 PM, Lisa Woodring <lisa.woodring@iglass.net> wrote:
>> On Tue, Nov 18, 2014 at 2:26 PM, André Warnier <aw@ice-sa.com> wrote:
>>> Lisa Woodring wrote:
>>> ...
>>>> In order to monitor
>>>> the availability of the HTTPS/AJP port (Apache-->Tomcat), our
>>>> monitoring software opens a port to verify that this works -- but then
>>>> does not follow that up with an actual request.  This happens every 2
>>>> minutes.
>>> ...
>>>
>>> This sounds like the perfect recipe for simulating a DOS attack.  Your
>>> monitoring system is forcing Tomcat to allocate a thread to process the
>>> request which should subsequently arrive on that connection, yet that
>>> request never comes; so basically this thread is wasted, until the
>>> ConnectionTimeout triggers (after 20 seconds, according to your HTTP
>>> connector settings).
>>>
>>> ...
>>>> The thread count grows over time (goes up to 130-150 threads after 2
>>>> hours).  Setting 'connectionTimeout' (as opposed to the default of
>>>> never timing out) does seems to help "some"
>>>
>>> Have you tried setting it shorter ? 20000 = 20000 ms = 20 seconds. That is
>>> still quite long if you think about a legitimate browser/application making
>>> a connection, and then sending a request on that connection.  Why would it
>>> wait so long ? A browser would never do that : it would open a connection to
>>> the server when it needs to send a request, and then send the request
>>> immediately, as soon as the connection is established.
>>>
>>> In other words : anything which opens a HTTP connection to your server, and
>>> then waits more than 1 or 2 seconds before sending a request on that
>>> connection, is certainly not a browser.
>>> And it probably is either a program designed to test or attack your server,
>>> or else a badly-designed monitoring system.. ;-)
>>>
>>
>> The monitoring software is going thru Apache to AJP connector in
>> Tomcat.  As I described, with the default of no timeout, the # of
>> threads were much higher.  I currently have the AJP connectionTimeout
>> set to 3 seconds.
>
> Actually, I received a little clarification on the monitoring software
> (I didn't write it).  What it's trying to test is that the AJP port
> itself is actually accepting connections.  With Apache in front in a
> production system, it could forward the actual request to one of
> several Tomcat boxes -- but we don't know which one from the outside.
> The monitoring software is trying to test -- for each Tomcat instance
> -- if it is accepting connections.  It used to send an "nmap" request,
> but now sends essentially a "tcp ping" -- gets a response & moves on.

In my case (homemade monitoring) i choosed to check mod_jk's log, after 
all mod_jk does indeed check the state of the ajp connector in tomcat.

Hope this helps.
[... ]


Bye,
Frederik

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message