httpd-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "King Holger (CI/AFP2)" <Holger.K...@de.bosch.com>
Subject [users@httpd] Apache2 and Tomcat load balancing | Timeouts: request, connection, cping/cpong
Date Fri, 03 May 2013 10:01:20 GMT
Dear Apache2 community,

having installed Apache2 2.4.3 in combination with:

 *   mod_proxy_balancer (not "mod_jk"!)
 *   mod_proxy_ajp
 *   mod_proxy
and the following virtual host configuration content:

<VirtualHost *:8000>
...
BalancerMember ajp://rb-wcmstc1.xx.xxxxx.xxx:8010 loadfactor=50 ping=5 retry=10 route=rb-wcmstc1
connectiontimeout=5 timeout=300
BalancerMember ajp://rb-wcmstc2.xx.xxxxx.xxx:8010 loadfactor=50 ping=5 retry=10 route=rb-wcmstc2
connectiontimeout=5 timeout=300
...
<VirtualHost>

to allow and setup a Tomcat 6 load balancing infrastructure, we regularly see the above configured
timeouts in seconds (see http://httpd.apache.org/docs/2.2/mod/mod_proxy.html#proxypass):
- request-timeout (300 seconds)
- connection-timeout (5 seconds)
- cping/cpong-timeout (5 seconds)
being exceeded. The Apache2 error log proves that:

 1.  [Mon Apr 08 18:23:16.954749 2013] [proxy_ajp:error] [pid 5030:tid 48] (70007)The timeout
specified has expired: AH01030: ajp_ilink_receive() can't receive header
 2.  [Mon Apr 08 18:25:28.422599 2013] [proxy_ajp:error] [pid 6623:tid 52] (70007)The timeout
specified has expired: [client 10.25.131.11:28706] AH00897: cping/cpong failed to xx.xx.xx.xxx:8010
(rb-wcmstc1.xx.xxxxx.xxx), referer: https://rb-xxxxx.xx.xxxxx.xxx/fs4webedit/message.jsp

 1.  might be logged when a request took longer >300 seconds
 2.  might be logged when ping request took longer >5 seconds
Taking a look at the HTTPS Access logs, we see for:

 1.  a request really taking 300 sec. It leads to the above error log entry at 18:23:16 -
five minutes later:
xx.xx.xx.xxx - - [08/Apr/2013:18:18:16 +0200] "POST /fs4webedit/Dispatcher HTTP/1.1" 302 295
"..." "F0FDAFA023B323A62813598C985C3AB1.rb-wcmstc1" "-" TLSv1 RC4-MD5 300004793
With an important BUG fix provided since Apache2 version 2.2.22 we have a solution in place
that prevents marking all AJP-workers (used for communication between Apache2 and Tomcat)
pointing to the Tomcat instance in ERROR state where the single long running request occurred.
This bugfix only works when setting the "ping" attribute for the BalancerMember (see above).
Details see ChangeLog enclosed: http://www.apache.org/dist/httpd/CHANGES_2.2.22 respectively
http://mail-archives.apache.org/mod_mbox/httpd-cvs/201201.mbox/%3C20120104201529.54EA823888D2@eris.apache.org%3E

Now, our questions are:

 *   what could provoke the 2nd error message: "AH01012: ajp_handle_cping_cpong: ajp_ilink_receive
failed" (a stop-the-world garbage collection on Tomcat side?)
 *   what are the consequences if that happens?
 *   how to prevent it (simply increasing the ping timeout?)
Best regards,
Holger King

Mime
View raw message