Mailing-List: contact users-help@tomcat.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Tomcat Users List" <users@tomcat.apache.org>
Received-SPF: pass (athena.apache.org: domain of anuragkapur@gmail.com
 designates 209.85.218.220 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:date:message-id:subject:from:to:content-type;
        b=q87rrvK+O7uOdl313/raB5m6QcEP0ZoibLyAinMwtQSxtPSLy5WNbNgzVXKNzLpLPx
         7mSJPNq7GFNqXynQAEL+n0zUnrqhMv2hFh1sDafQf2ZT8n5i4ClW6PzrXZUmYVLsTcg4
         5tFEdtV3uXLGg49t9pyveagrVGAXB7PnUjoBE=
MIME-Version: 1.0
Date: Wed, 10 Feb 2010 02:35:11 +0000
Message-ID: <2fb267f81002091835h534b04d8i43a761809414d234@mail.gmail.com>
Subject: Tomcat HTTP Connector Threads Hung
From: Anurag Kapur <anuragkapur@gmail.com>
To: Tomcat Users List <users@tomcat.apache.org>
Content-Type: multipart/alternative; boundary=00032555930a2f8e98047f35e2e7

--00032555930a2f8e98047f35e2e7
Content-Type: text/plain; charset=ISO-8859-1

*Problem Statement

*While load testing a web application, several tomcat http connector threads
get stuck. Ultimatelty all the available http connector threads get
exhausted and the application stops responding completely.

*Setup*

I have an apache http server in front of tomcat which proxies requests to
it. I am using mod_proxy_http module(I do not know the exact version. Is
there an easy way to find this out?) to proxy requests to tomcat.

Relevant snippet from apache httpd.conf that shows the proxy setting:

ProxyPass                 http://localhost:8080/ retry=0
ProxyPassReverse          http://myserver/

*Observations*

Since there is no explicit timeout set along with the proxy directive, it
assumes a default value of 300s. Thus, after 300s, if apache does not get
back a response from tomcat, it servers a HTTP 502 error code back to the
user.

However, the corresponding http connector thread does not terminate and
continues waiting for a response. As a result, with time a lot of http
connector threads get stuck and ultimately the whole pool gets exhausted.

Http connector settings from tomcat server.xml:
<Connector port="8080" maxHttpHeaderSize="8192"
               maxThreads="300" minSpareThreads="25" maxSpareThreads="77"
               enableLookups="false" redirectPort="8443" acceptCount="256"
               connectionTimeout="300000" disableUploadTimeout="true" />

*Questions*

1. What does the connectionTimeout attribute in the tomcat http connector
do? Does it not terminate the thread if it is still processing the request
once the timeout expires? I assume it does NOT and that is why I am seeing
this problem.

2. Can some other configuration be added that forces the http connector
thread to terminate after a certain time interval has elapsed?

3. Any other thoughts or advice to solve this problem? I know I should
investigate why the http connector thread does not return a response even in
as long as 300s, but assuming there is a bug in the application code
shouldn't there be a settings in tomcat that prevents such application bugs
from exhausting all tomcat connector threads?

*Other System Information*

Server version: Apache Tomcat/5.5.27
Server built:   Aug 28 2008 10:08:26
Server number:  5.5.27.0
OS Name:        SunOS
OS Version:     5.10
Architecture:   x86
JVM Version:    1.5.0_14-b03
JVM Vendor:     Sun Microsystems Inc.

Apache Server version: Apache/2.2.9 (Unix)

Your thoughts and advice on this would be much appreciated.

Thanks
Anurag

------------------------------------------------------------------
Anurag Kapur
Associate - Technology,
Sapient Corporation.

http://www.linkedin.com/in/anuragkapur

http://www.google.com/profiles/anuragkapur
------------------------------------------------------------------

--00032555930a2f8e98047f35e2e7--