tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Richard S. Huntrods" <huntr...@nucleus.com>
Subject Re: Would like to monitor memory use offline
Date Tue, 12 Aug 2008 15:54:06 GMT
> Richard,
>
> Richard S. Huntrods wrote:
> | Everything is local except the pooled connections, so I would say this
> | is the problem. This code was originally written before tomcat had good
> | connection pools, and so I had to write my own. The pool is basically a
> | vector of connections.
>
> I wouldn't want to blame your connection pool just yet; I'm just saying
> that it is a possibility. The open-source connection pools have a /lot/
> of users and a /lot/ of eyes looking at them, and tend to be very
> stable. Your in-house connection pool may not have been as thoroughly
> tested.
My in-house connection pool was based on code examples from around 2001 
- when I first implemented the system. My database classes were from the 
same time, hence the "almost correct" nature of the code (close 
statements, finally clause). It all worked without incident until this 
July, when I was required to upgrade an aging server and also upgraded 
all of the packages. We had been using the original mmjb.jar version 
(predating when MySQL took over the connector/J project).

It took a while to trace the memory leaks to the connector, and then it 
was quick work to find the version split where the leak started. (see 
prior posts). In that time I had been reading as much as possible about 
Tomcat memory use, leaks, and finally stuff about the connector.


>
> | At a more fundamental level, I still don't think the new connector is
> | working correctly (and I've read many of the forum discussions about
> | this...).
>
> Has anyone mentioned the behavior you are observing? Were those posts
> anything but idle speculation, or has anyone come to an actual
> conclusion about the cause of the errors?
The majority of the posts I'm referring to occurred in 2004. Many people 
were noticing memory leaks with the connectors, and wanting to call them 
Bugs. The database folks and connector authors fiercly defended their 
work and stated the problem was always due to missing close() statements.

The discussion then turned toward "misinterpretation of the JDBC 
specification" - where the people complaining about the leak insisted 
the spec. required the connector to release all memory objects as soon 
as the statement went out of scope and the connector people saying "No - 
you must call close()".

SO I put the appropriate close() for statement in my code - but to no 
effect. With the exception of my connection, my code now matches what 
"is correct".
>
> | If I close the resultset AND the statement, then the connector
> | should release all the objects created by those two. The connection is,
> | after all, just a pipe between the database and the java code. The
> | connector should not (IMO) be hanging on to statement or resultset
> | objects just because the connection is still in existance.
>
> I completely agree. Two things that I can think of that might be causing
> problems with the Connector itself:
>
> 1. ResultSetMetadata -- use of the metadata methods can cause additional
> queries to be executed, which means more ResultSet objects, Fields, etc.
> I didn't see any use of this in your sample code, so I suspect this is
> not the issue.
Actually, it is. I use a RSMD object in all my queries. I don't close 
it. - MAYBE THAT's THE PROBLEM. I will test and report...

>
> 2. Client-side PreparedStatement caching. Have you enabled any of the
> caching options provided by the driver? I'm guessing not, since you were
> using a 3.x driver before, which IIRC does not support such caching.
Nope. No caching at all.

>
> | Still - this is most clearly NOT my problem as I can verify my code is
> | working "normally" (i.e. no exceptions happening) and thus properly
> | closing both the resultset and the statement.
>
> Agreed.
>
> | For fun I put try-catch blocks with additional resultset and statemtent
> | calls to close in the finally block and it made no difference to the
> | memory leak.
>
> Right. I would not have predicted that you would change anything by
> doing this -- it's just a good practice to get into in the event that
> exceptions /do/ start cropping up.
Agreed.
>
> | Yea - the code was developed a long while ago. There's also a single
> | version that can run under Tomcat or as offline processes and I don't
> | want to have to write two different database mechanisms to support the
> | application outside of Tomcat.
>
> Tomcat uses JNDI DataSources to provide connection pooling. This is very
> easy to do outside of Tomcat if you are interested. Another option would
> be gut your existing CP implementation and replace it with pass-through
> calls to a library like commons-dbcp. This is exactly what I did with a
> project that I inherited a few years ago and many, many JDBC-related
> problems went away (more to do with connection leakage than memory 
> leakage).
Sounds like a plan - if ever I have time to do this. ;-)
>
> My last suggestion would be to use a real memory profiler. These tools
> can typically tell you exactly where in your code a particular object
> was instantiated. After you run your program for a while, inspect the
> heap (and all those Field objects) and see where they are being created.
> You may find that you are looking at fixing the wrong method in your own
> code -- or that the driver itself is leaking Field objects (which would
> be unlikely, but certainly possible).
yes - that is the logical next step.
>
> Good luck,
> - -chris 

---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message