tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Kumar <dku...@telestar.de>
Subject AW: afs3-rmtsys: connections keept open
Date Wed, 16 Jan 2013 20:41:25 GMT
Hey,

thanks for replying..

I forgot to mention that we have a Apache being a loadbalancer in front of these two tomcats.
The Apache uses port 8080.

Let me start with André Warnier:

Both of the tomcats are "synchronising" them self. The send some serialized objects via http
to each other. And both of them getting some file from SMB shares. But I can't imagine that
might be the problem?
I'm wondering why the tcp connections with state "CLOSE_WAIT" doesn't get closed. 

Here are the requested connectors

tomcat1:

    <Connector port="7080" protocol="HTTP/1.1" 
               connectionTimeout="20000" 
               redirectPort="8443" />
<Connector executor="tomcatThreadPool"
               port="7080" protocol="HTTP/1.1" 
               connectionTimeout="20000" 
               redirectPort="8443" />
    <Connector port="8443" protocol="HTTP/1.1" SSLEnabled="true"
               maxThreads="150" scheme="https" secure="true"
               clientAuth="false" sslProtocol="TLS" />
<Connector port="7009" protocol="AJP/1.3" redirectPort="8443" />

tomcat2:
<Connector port="9080" protocol="HTTP/1.1" 
               connectionTimeout="20000" 
               redirectPort="9443" />
<Connector executor="tomcatThreadPool"
               port="8080" protocol="HTTP/1.1" 
               connectionTimeout="20000" 
               redirectPort="8443" />
    <Connector port="8443" protocol="HTTP/1.1" SSLEnabled="true"
               maxThreads="150" scheme="https" secure="true"
               clientAuth="false" sslProtocol="TLS" />
<Connector port="9009" protocol="AJP/1.3" redirectPort="9443" />


Yeah I'm sure its port 7090, as you can see above.. :-)
7009 is used by AJP!?

As I had to restart the server, I don't know who was the owner of that PID.
Right now we do have 2 or 3 of:
java    9495 tomcat  201u  IPv6            1866645      0t0     TCP localhost:afs3-rmtsys->localhost:55116
(ESTABLISHED)

If I run ulimit -n with the tomcat user I get: 1024. But I did set the max open files at the
startscript for each tomcat (init.d) with: 
ulimit -Hn 4096
ulimit -Sn 4096

And if I check the files for the current tomcat process (cat proc/9180/limits while 9180 is
one of my tomcat PIDs) I get:
Max open files            4096                 4096                 files

Thanks again..





Mit freundlichen Grüßen
David Kumar  
Softwareentwickler, B. Sc.
Abteilung Infotech - Interaktiv 
TELESTAR-DIGITAL GmbH
Am Weiher 14
D-56766 Ulmen

http://www.telestar.de/




-----Ursprüngliche Nachricht-----
Von: Christopher Schultz [mailto:chris@christopherschultz.net] 
Gesendet: Mittwoch, 16. Januar 2013 19:14
An: Tomcat Users List
Betreff: Re: afs3-rmtsys: connections keept open

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

David,

On 1/16/13 10:52 AM, David Kumar wrote:
> We got some serious trouble with our tomcats. Basically we using
> two 7.0.12 tomcats on a single Debian 6.0. One is running on port
> 7090 and the other on 9080. Time after time both tomcats are
> crashing. I found too many open files in tomcatlogfiles.

Bummer.

> Both Servers are running with one user (TOMCAT).

Can you post the <Connector> parts of your server.xml files (from both
Tomcat configurations)?

Also, what does `ulimit -n` return from an otherwise unmodified shell
running as the user who owns Tomcat processes?

> After changing max open files for that particular user I was able
> to get more than 1024 open files. Currently I'm checking log files
> every few minutes. With lsof|grep tomcat|wc -l I'm counting the
> open files by user. Usually there are around 600-700 open files.
> But sometimes the count grows and we have more than 1200 open
> files.

You'll get file handled opened for both real files (like /tmp/foo,
etc.) and for socket connections. If you aren't careful, you can have
a configuration that will work for a while bit tip-over under load.

> When ever the server does have so many open files I checked: lsof
> -u tomcat an figured out there are many connection waiting to be
> closed. The connections are looking like this:
> 
> java    22312 tomcat  153u  IPv6            1508517      0t0
> TCP localhost:afs3-rmtsys->localhost:50127 (CLOSE_WAIT)

Are you sure your port is 7090 and not 7009? Is 22312 the right
process id for Tomcat? The well-known port number for the
"afs3-rmtsys" service is 7009. It would be odd, though, for your Java
process to be calling *out* from a port like that ... usually you get
a high-numbered port when making an outgoing connection. Who owns
localhost:50127 in that case?

- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with undefined - http://www.enigmail.net/

iEYEAREIAAYFAlD27f4ACgkQ9CaO5/Lv0PD7RQCfd7PaA5eY4pANZY6NYhGB3OmN
5+4AoJ9GihEA9NZr2BoCIARfBTaTqs8T
=x9/z
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message