tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pid *" <...@pidster.com>
Subject Re: Tomcat 6.0.35-SocketException: Too many open files issue with
Date Fri, 27 Jan 2012 09:03:43 GMT
On 27 Jan 2012, at 05:32, gnath <gautam_exquisite@yahoo.com> wrote:

> Hello Chris,
>
>
> After seeing the initial connection pool issue, i started searching online for help and
i found this article :
> http://vigilbose.blogspot.com/2009/03/apache-commons-dbcp-and-tomcat-jdbc.html
> so, i thought may be tomcat's jar would bring some improvement. by the way, we had commons-dbcp-1.3.jar.
Do you recommend upgrading to newer commons dbcp jar instead of using tomcat-jdbc.jar.

Tomcat ships with a DBCP implementing of its own.

How and where are you defining the database?


p



>
> Just because we are running tomcat-6.0.35, it did not come with tomcat-jdbc.jar, so we
downloaded the 1.1.1 version or jar and dropped in WEB-INF/lib and started using it.
>
>
> I agree what you are saying about leaking the connection and will plan to set the logAbandoned
flag as you suggested.
>
>
> However, i was about to file a new issue but would like to describe here as well. So
we have 2 servers running tomcat (same code, same configuration). After we replaced tomcat-jdbc.jar
and added 'removeAbandoned' flag to true, one of the servers is doing great (ofcourse i agree
that pool is cleaning up the mess), but we saw one new issue on the second server. it hasn't
been releasing the connections and was consistently growing slowly. So i collected thread
dump and i saw a deadlock :
>
> Found one Java-level deadlock:
> =============================
> "catalina-exec-1":
>   waiting to lock monitor 0x000000005d7944b8 (object 0x00000005bd522568, a com.mysql.jdbc.Connection),
>   which is held by "[Pool-Cleaner]:Tomcat Connection Pool[1-1015483951]"
> "[Pool-Cleaner]:Tomcat Connection Pool[1-1015483951]":
>   waiting to lock monitor 0x000000005dcdea28 (object 0x00000005bd659ce8, a com.mysql.jdbc.ResultSet),
>   which is held by "catalina-exec-1"
>
> Java stack information for the threads listed above:
> ===================================================
> "catalina-exec-1":
>         at com.mysql.jdbc.Connection.getCharsetConverter(Connection.java:3177)
>         - waiting to lock <0x00000005bd522568> (a com.mysql.jdbc.Connection)
>         at com.mysql.jdbc.Field.getStringFromBytes(Field.java:583)
>         at com.mysql.jdbc.Field.getName(Field.java:487)
>         at com.mysql.jdbc.ResultSet.buildIndexMapping(ResultSet.java:593)
>         at com.mysql.jdbc.ResultSet.findColumn(ResultSet.java:926)
>         - locked <0x00000005bd659ce8> (a com.mysql.jdbc.ResultSet)
>         at com.mysql.jdbc.ResultSet.getInt(ResultSet.java:2401)
>
> "[Pool-Cleaner]:Tomcat Connection Pool[1-1015483951]":
>         at com.mysql.jdbc.ResultSet.close(ResultSet.java:736)
>         - waiting to lock <0x00000005bd659ce8> (a com.mysql.jdbc.ResultSet)
>         at com.mysql.jdbc.Statement.realClose(Statement.java:1606)
>         - locked <0x00000005bd522568> (a com.mysql.jdbc.Connection)
>         - locked <0x00000005bd5e81c0> (a com.mysql.jdbc.ServerPreparedStatement)
>         at com.mysql.jdbc.PreparedStatement.realClose(PreparedStatement.java:1703)
>         at com.mysql.jdbc.ServerPreparedStatement.realClose(ServerPreparedStatement.java:901)
>         - locked <0x00000005bd525ba0> (a java.lang.Object)
>         - locked <0x00000005bd522568> (a com.mysql.jdbc.Connection)
>         - locked <0x00000005bd5e81c0> (a com.mysql.jdbc.ServerPreparedStatement)
>         at com.mysql.jdbc.Connection.closeAllOpenStatements(Connection.java:2126)
>         at com.mysql.jdbc.Connection.realClose(Connection.java:4422)
>         at com.mysql.jdbc.Connection.close(Connection.java:2098)
>         - locked <0x00000005bd522568> (a com.mysql.jdbc.Connection)
>         at org.apache.tomcat.jdbc.pool.PooledConnection.disconnect(PooledConnection.java:320)
>
>
>
> Please help us on this. Could it be a problem with tomcat-jdbc.jar?
>
> Thanks
> -G
>
>
> ________________________________
> From: Christopher Schultz <chris@christopherschultz.net>
> To: Tomcat Users List <users@tomcat.apache.org>
> Sent: Thursday, January 26, 2012 9:41 AM
> Subject: Re: Tomcat 6.0.35-SocketException: Too many open files  issue with
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> G,
>
> On 1/25/12 11:53 PM, gnath wrote:
>> As you have suggested, i started collecting the thread dumps
>
> Thread dumps will set you free. Well, not really. Instead, they will
> tell you where your webapp is breaking, which usually means more work
> for you. But at least the hard part is done: finding out what's breaking.
>
>> when it happened again and we saw some kind of DBCP Connection pool
>> issues leading to 'Too Many open files' issue.
>
> That will definitely do it.
>
>> So we decided to replace the commons DBCP with tomcat-jdbc.jar
>> (with same configuration properties).
>
> Why?
>
>> After this change, it seemed for few hours but started seeing in
>> the logs where the Connection Pool jar could not give any
>> connections and seems to be all the connections are busy. So we
>> went ahead and added a configuration property
>> 'removeAbandoned=true' in our Datasource configuration.
>
> I would go back to DBCP unless you think you need to switch for some
> reason.
>
> I suspect you are leaking database connections and don't have a
> suitable timeout for removal of "lost" database connections (or maybe
> didn't have that set up in the first place).
>
> You really need to enable "logAbandoned" so you can find out where
> your connection leaks are, and fix them. In development, set
> maxActive="1" and leave it there, forever. Also, set
> logAbandoned="true" and always run like that in development. Running
> like that in production isn't a bad idea, either.
>
>> We are still watching the performance and the server behavior
>> after these changes. Will keep you posted on how things will turn
>> out or if i see any further issues.
>
> I suspect you are still leaking connections, but your pool is now
> silently cleaning-up after the mess your webapp is making. Instrument
> your pool. Fix your leaks.
>
> - -chris
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAk8hkDkACgkQ9CaO5/Lv0PCxFgCgs+EiV/CNjmCNekeDwKHgnNtZ
> 5LYAoKZUkIAJOK0eItkoHBF3wScK9lQf
> =AyL4
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message