tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Frederik Nosi <frederik.n...@postecom.it>
Subject Re: High thread count & load on Tomcat8 when accessing AJP port with no request
Date Thu, 20 Nov 2014 11:42:17 GMT
On 11/19/2014 09:27 PM, Lisa Woodring wrote:
>>> Actually, I received a little clarification on the monitoring software
>>> (I didn't write it).  What it's trying to test is that the AJP port
>>> itself is actually accepting connections.  With Apache in front in a
>>> production system, it could forward the actual request to one of
>>> several Tomcat boxes -- but we don't know which one from the outside.
>>> The monitoring software is trying to test -- for each Tomcat instance
>>> -- if it is accepting connections.  It used to send an "nmap" request,
>>> but now sends essentially a "tcp ping" -- gets a response & moves on.
>>
>> In my case (homemade monitoring) i choosed to check mod_jk's log, after all
>> mod_jk does indeed check the state of the ajp connector in tomcat.
>>
>> Hope this helps.
>> [... ]
>
> Thanks for the idea.  Can you tell me what you specifically look for
> in the "mod_jk_log" file?  Do you look for the presence of something?
> or the absence of something?

grep out cping,

> I only see 'negative' events in the logfile.  For example,
> "all endpoints are disconnected, detected by connect check(1),
> cping(0), send(0)"
> which evidently, is when Tomcat releases a connection on its end.
> (I set JkLogLevel = DEBUG, but still don't see any messages that look
> like what I would want...)
Just ignore the cping part. I categorize the failure modes in two, 
client error (user closes the browser window or is slow), example:


[Thu Nov 20 10:19:36 2014] [29858:1626331456] [info] 
service::jk_lb_worker.c (1388): service failed, worker p3 is in local 
error state
[Thu Nov 20 10:19:36 2014] [29858:1626331456] [info] 
service::jk_lb_worker.c (1407): unrecoverable error 200, request failed. 
Client failed in the middle of request, we can't recover to another 
instance.
[Thu Nov 20 10:19:36 2014] [29858:1626331456] [info] 
jk_handler::mod_jk.c (2611): Aborting connection for worker=worker_p

Or server error, can be because of timeout (backend too busy):

[Thu Nov 20 10:19:54 2014] [31475:1317062976] [error] 
ajp_get_reply::jk_ajp_common.c (2020): (p7) Timeout with waiting reply 
from tomcat. Tomcat is down, stopped or network problems (errno=110)
[Thu Nov 20 10:19:54 2014] [31475:1317062976] [info] 
ajp_service::jk_ajp_common.c (2540): (p7) sending request to tomcat 
failed (recoverable), because of reply timeout (attempt=1)
[Thu Nov 20 10:19:54 2014] [31475:1317062976] [error] 
ajp_service::jk_ajp_common.c (2559): (p7) connecting to tomcat failed.

Another server error is connection refused, when the backend is extra 
busy (on linux net.ipv4.tcp_max_syn_backlog sockets waiting on the tcp 
stack) or tomcat is down. I dont have an example of this right now though


Anyway, this way you use mod_jk's logic instead of having to create an 
ad hoc one. This is at JkLogLevel notice, no need to enable debug.



---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message