Mailing-List: contact users-help@tomcat.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Tomcat Users List" <users@tomcat.apache.org>
Received-SPF: neutral (athena.apache.org: local policy)
Message-ID: <50F6EE00.2060302@christopherschultz.net>
Date: Wed, 16 Jan 2013 13:14:24 -0500
From: Christopher Schultz <chris@christopherschultz.net>
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8;
 rv:17.0) Gecko/17.0 Thunderbird/17.0
MIME-Version: 1.0
To: Tomcat Users List <users@tomcat.apache.org>
Subject: Re: afs3-rmtsys: connections keept open
References: <08E5CE5F1D6E3145B4FA31AA948C54FE2B4ABEF4@x-dag01>
In-Reply-To: <08E5CE5F1D6E3145B4FA31AA948C54FE2B4ABEF4@x-dag01>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

David,

On 1/16/13 10:52 AM, David Kumar wrote:
> We got some serious trouble with our tomcats. Basically we using
> two 7.0.12 tomcats on a single Debian 6.0. One is running on port
> 7090 and the other on 9080. Time after time both tomcats are
> crashing. I found too many open files in tomcatlogfiles.

Bummer.

> Both Servers are running with one user (TOMCAT).

Can you post the <Connector> parts of your server.xml files (from both
Tomcat configurations)?

Also, what does `ulimit -n` return from an otherwise unmodified shell
running as the user who owns Tomcat processes?

> After changing max open files for that particular user I was able
> to get more than 1024 open files. Currently I'm checking log files
> every few minutes. With lsof|grep tomcat|wc -l I'm counting the
> open files by user. Usually there are around 600-700 open files.
> But sometimes the count grows and we have more than 1200 open
> files.

You'll get file handled opened for both real files (like /tmp/foo,
etc.) and for socket connections. If you aren't careful, you can have
a configuration that will work for a while bit tip-over under load.

> When ever the server does have so many open files I checked: lsof
> -u tomcat an figured out there are many connection waiting to be
> closed. The connections are looking like this:
> 
> java    22312 tomcat  153u  IPv6            1508517      0t0
> TCP localhost:afs3-rmtsys->localhost:50127 (CLOSE_WAIT)

Are you sure your port is 7090 and not 7009? Is 22312 the right
process id for Tomcat? The well-known port number for the
"afs3-rmtsys" service is 7009. It would be odd, though, for your Java
process to be calling *out* from a port like that ... usually you get
a high-numbered port when making an outgoing connection. Who owns
localhost:50127 in that case?

- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with undefined - http://www.enigmail.net/

iEYEAREIAAYFAlD27f4ACgkQ9CaO5/Lv0PD7RQCfd7PaA5eY4pANZY6NYhGB3OmN
5+4AoJ9GihEA9NZr2BoCIARfBTaTqs8T
=x9/z
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org