tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Filip Hanik Mailing Lists <devli...@hanik.com>
Subject Re: Discrepancy between Tomcat's connection pool and tomcat's report on memory leaks
Date Wed, 04 Apr 2012 17:03:02 GMT
The real fix to your problem should have been

<Resource type="javax.sql.DataSource" closeMethod="close" .../> (available in Tomcat
7)

If I understand the problem you're having is two fold
1. you see reports about memory leaks
"started a thread named [H2 File Lock Watchdog" you should be able to fix this by setting
initialSize>0 so that the thread is started using the main class loader when tomcat starts,
not when an app starts using the pool.
(Tomcat 7's jdbc-pool does this automatically from 7.0.27 onwards, regardless of configuration)

2. The connection pool is not closed properly, again, this is addressed in the top of this
email.

letting the pool eventually shrink to 0, is merely a work around, but I realize it will NOT
work. H2 is starting it's own thread, there is no way around this, unless there is way to
shut down this thread. Maybe this shuts down if all connections are closed.

Filip

----- Original Message -----
> From: "Hermes Flying" <flyinghermes@yahoo.com>
> To: "Filip Hanik Mailing Lists" <devlists@hanik.com>, "Tomcat Users List" <users@tomcat.apache.org>
> Sent: Wednesday, April 4, 2012 10:56:10 AM
> Subject: Re: Discrepancy between Tomcat's connection pool and tomcat's report on memory
leaks
> 
> 
> 
> Ok. So how is the configuration parameter fixing my problem? Would
> you please explain this to me to undestand how this works?
> 
> 
> 
> 
> 
> From: Filip Hanik Mailing Lists <devlists@hanik.com>
> To: Tomcat Users List <users@tomcat.apache.org>; Hermes Flying
> <flyinghermes@yahoo.com>
> Sent: Wednesday, April 4, 2012 7:50 PM
> Subject: Re: Discrepancy between Tomcat's connection pool and
> tomcat's report on memory leaks
> 
> no, that would happen if you set maxIdle=0, not minIdle
> 
> ----- Original Message -----
> > From: "Hermes Flying" < flyinghermes@yahoo.com >
> > To: "Tomcat Users List" < users@tomcat.apache.org >
> > Sent: Wednesday, April 4, 2012 10:45:24 AM
> > Subject: Re: Discrepancy between Tomcat's connection pool and
> > tomcat's report on memory leaks
> > 
> > But if I set 'minIdle=0' all the connections would close
> > imediatelly,
> > right?
> > So why would I need a connection pool in the first place if I do
> > this?
> > 
> > 
> > ________________________________
> > From: Filip Hanik Mailing Lists < devlists@hanik.com >
> > To: Tomcat Users List < users@tomcat.apache.org >
> > Sent: Wednesday, April 4, 2012 7:28 PM
> > Subject: Re: Discrepancy between Tomcat's connection pool and
> > tomcat's report on memory leaks
> > 
> > just set minIdle=0 and enable the eviction process to take care of
> > it.
> > 
> > Filip
> > 
> > 
> > ----- Original Message -----
> > > From: "Hermes Flying" < flyinghermes@yahoo.com >
> > > To: "Daniel Mikusa" < dmikusa@vmware.com >
> > > Cc: users@tomcat.apache.org
> > > Sent: Wednesday, April 4, 2012 9:53:30 AM
> > > Subject: Re: Discrepancy between Tomcat's connection pool and
> > > tomcat's report on memory leaks
> > > 
> > > >>Which is indicating that the application deployed to
> > > >>"/GeneralApplication" is creating a thread named "H2 Log Writer
> > > >>>>GENERICAPPLICATION" and never stopping it. I do not believe
> > > >>that this would be associated with the pool created by Tomcat
> > > >>as
> > > >>that would >>be created as a global resource available to all
> > > >>applications and not specific to "/GeneralApplication".
> > > >>I can't really say a whole lot more about what is going on,
> > > >>just
> > > >>that this strikes me as odd. If it were my app, I'd dig into
> > > >>this
> > > >>more and see what is >>creating that thread. Possibly with a
> > > >>profiler or thread dumps.
> > > 
> > > What is happening here (as I undestand it is the following):
> > > 1) My application uses Tomcat's connection pool from the data
> > > source
> > > 2) My application uses H2 as a file database. H2 starts back
> > > end(s)
> > > thread which is about logging (H2 logging) and possibly other
> > > tasks
> > > and that is the thread that catalina complaints about
> > > 3) H2 remains open as long as there are open connections and is
> > > closed when the last connection is released
> > > 4) My application uses Tomcat's connection pool and as a result
> > > Tomcat "decides" when the data source should be disposed or when
> > > connections are released (not my code).
> > > 5) I shutdown Tomcat.
> > > 6) The connection pool of Tomcat has still connections (e.g.
> > > idle)
> > > and so H2 does not shutdown. Hence it's lock file and related
> > > background threads are still running.
> > > 7) Tomcat shuts down my application but does not dispose off the
> > > connection pool (yet). I believe this happens because of the way
> > > Tomcat is being shut down (as sequence of actions I mean).
> > > 8) Since the connection pool has not been disposed, there are
> > > still
> > > connections and as a result H2 is still running
> > > 9) Tomcat's memory leak detection mechanism detects H2 threads
> > > started from GenericApplication which uses H2 and gives the
> > > report
> > > in catalina.out. So Tomcat gives the report first and then
> > > proceeds
> > > with the shutdown (and this perhaps is not correct? I can't say)
> > > 10) Tomcat eventually shuts down.
> > > 
> > > Is this scenario as I describe it, wrong?
> > > Should I somehow have configured the datasource to be
> > > "associated"
> > > with my web application instead of a generic data source? Why?
> > > Should I have shut the data source myself? How?
> > > If all what I describe is reasonable this seems to me like a
> > > "racing"
> > > condition. How should I mediate this?
> > > 
> > > Thank you for your time.
> > > 
> > > 
> > > ________________________________
> > > From: Daniel Mikusa < dmikusa@vmware.com >
> > > To: Hermes Flying < flyinghermes@yahoo.com >
> > > Sent: Wednesday, April 4, 2012 5:14 PM
> > > Subject: Re: Discrepancy between Tomcat's connection pool and
> > > tomcat's report on memory leaks
> > > 
> > > ----- Original Message -----
> > > > 
> > > > 
> > > > What I do is get the dataSource inside a ServletContextListener
> > > > and
> > > > save it in servlet context (as part of a DAO Factory):
> > > > 
> > > > public void contextInitialized(ServletContextEvent sce) {
> > > > 
> > > > ServletContext context = sce.getServletContext();
> > > > DataSource dataSource = null;
> > > > try {
> > > > dataSource = (DataSource) new
> > > > InitialContext().lookup("java:/comp/env/jdbc/GenericDataSource");
> > > > context.setAttribute("daogenericfactory",
> > > > DAOGenericFactory.getInstance(dataSource));
> > > > 
> > > > } catch (NamingException e) {
> > > > e.printStackTrace();
> > > > }
> > > > 
> > > > }
> > > > 
> > > > 
> > > > The DAOGenericFactory just returns specific DAO objects passing
> > > > in
> > > > their constructor the data source.
> > > > Each DAO instance has an member variable of DataSource and for
> > > > action
> > > > to the database I do in order to get a connection:
> > > > 
> > > > public Connection getConnection() throws Exception{
> > > > return dataSource.getConnection();
> > > > }
> > > > 
> > > > 
> > > > So I use the Connection this way and I always close the
> > > > connections
> > > > in order to be returned back to the pool.
> > > > The only thing perhaps to mention is that I create new DAO
> > > > objects
> > > > passing this data source (I don't reuse instances of DAO).
> > > > So I don't have a pool of my own. I just use DAO pattern .
> > > > This is happening in my web application GeneralApplication
> > > > Is this information adequate/clear?
> > > 
> > > Definitely clear. It just doesn't seem to match up with the
> > > output
> > > from the log files. You're seeing
> > > 
> > > SEVERE: The web application [/GeneralApplication] appears to
> > > have
> > > started a thread named [H2 Log Writer GENERICAPPLICATION] but
> > > has
> > > failed to stop it. This is very likely to create a memory leak.
> > > 
> > > Which is indicating that the application deployed to
> > > "/GeneralApplication" is creating a thread named "H2 Log Writer
> > > GENERICAPPLICATION" and never stopping it. I do not believe that
> > > this would be associated with the pool created by Tomcat as that
> > > would be created as a global resource available to all
> > > applications
> > > and not specific to "/GeneralApplication".
> > > 
> > > I can't really say a whole lot more about what is going on, just
> > > that
> > > this strikes me as odd. If it were my app, I'd dig into this more
> > > and see what is creating that thread. Possibly with a profiler or
> > > thread dumps.
> > > 
> > > 
> > > > Also how did you test it?
> > > 
> > > I took a stock Tomcat 7 instance, added the <Resource/> tag you
> > > specified to server.xml and added the <ResourceLink/> tag you
> > > specified to context.xml. I then deployed a test app that I have,
> > > it's a simple Grails app that I routinely use to test database
> > > issues. It just uses the JNDI connection specified to do some
> > > basic
> > > database crud.
> > > 
> > > 
> > > >Did you use H2?
> > > 
> > > Yes. h2-1.3.161.jar
> > > 
> > > 
> > > >Because I don't get these errors in catalina.out intermittently
> > > >but
> > > >always.
> > > 
> > > No errors in any of my log files. I tested stopping the Tomcat
> > > instance, restarting it and undeploying the application.
> > > 
> > > Dan
> > > 
> > > 
> > > 
> > > > 
> > > > Thank you
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > From: Daniel Mikusa < dmikusa@vmware.com >
> > > > To: Hermes Flying < flyinghermes@yahoo.com >
> > > > Sent: Wednesday, April 4, 2012 4:25 PM
> > > > Subject: Re: Discrepancy between Tomcat's connection pool and
> > > > tomcat's report on memory leaks
> > > > 
> > > > ----- Original Message -----
> > > > > 
> > > > > 
> > > > > Hi Dan,
> > > > > 
> > > > > 
> > > > > The following entry is in server.xml
> > > > > 
> > > > > <Resource name="jdbc_GENERIS_RS" auth="Container"
> > > > > type="javax.sql.DataSource"
> > > > > driverClassName="org.h2.Driver"
> > > > > url="jdbc:h2:file:/opt/en/repos/tomcat/webapps/GeneralApplication/db/internaldatabase;SCHEMA=genericschema"
> > > > > username="root" password="sa"
> > > > > maxActive="20" maxIdle="10" maxWait="-1"
> > > > > />
> > > > > 
> > > > > 
> > > > > And the following entry is under
> > > > > $TOMCAT_HOME/conf/context.xml
> > > > > 
> > > > > <ResourceLink name="jdbc/GenericDataSource"
> > > > > global="jdbc_GENERIS_RS"
> > > > > type="javax.sql.DataSource"/>
> > > > > 
> > > > > 
> > > > > 
> > > > > Do you need more details than these?
> > > > > 
> > > > > 
> > > > 
> > > > Seems a little weird. The message is indicating that the
> > > > threads
> > > > which have not stopped reside in the application
> > > > "[/GeneralApplication]". This would seem to imply that the
> > > > connection pool was created by the application. The
> > > > configuration
> > > > that you've specified here would have the container creating
> > > > the
> > > > connection pool, which shouldn't trigger messages like this.
> > > > 
> > > > Are you sure your application is not creating a connection
> > > > pool?
> > > > 
> > > > As a side note, I tried the configuration you mentioned in a
> > > > local
> > > > Tomcat instance and it worked fine, no errors.
> > > > 
> > > > Dan
> > > > 
> > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > From: Daniel Mikusa < dmikusa@vmware.com >
> > > > > To: Tomcat Users List < users@tomcat.apache.org >
> > > > > Sent: Wednesday, April 4, 2012 3:35 PM
> > > > > Subject: Re: Discrepancy between Tomcat's connection pool and
> > > > > tomcat's report on memory leaks
> > > > > 
> > > > > 
> > > > > 
> > > > > ----- Original Message -----
> > > > > > Hi,
> > > > > > 
> > > > > > I am using Tomcat 7.0.25 in a Linux machine
> > > > > > 
> > > > > > I am using Tomcat's connection pool
> > > > > > (org.apache.tomcat.dbcp.dbcp.BasicDataSource). As database
> > > > > > I
> > > > > > am
> > > > > > using H2 as a file database.
> > > > > 
> > > > > Where are you defining the connection pool? Can you include
> > > > > your
> > > > > configuration?
> > > > > 
> > > > > Dan
> > > > > 
> > > > > 
> > > > > > All is ok, but I have the following problem.
> > > > > > On shutdown of Tomcat I see in catalina.out:
> > > > > > 
> > > > > > 
> > > > > > SEVERE: The web application [/GeneralApplication] appears
> > > > > > to
> > > > > > have
> > > > > > started a thread named [H2 Log Writer GENERICAPPLICATION]
> > > > > > but
> > > > > > has
> > > > > > failed to stop it. This is very likely to create a memory
> > > > > > leak.
> > > > > > Apr 4, 2012 2:32:54 PM
> > > > > > org.apache.catalina.loader.WebappClassLoader
> > > > > > clearReferencesThreads
> > > > > > SEVERE: The web application [/GeneralApplication] appears
> > > > > > to
> > > > > > have
> > > > > > started a thread named [H2 File Lock Watchdog
> > > > > > /opt/en/repos/tomcat/webapps/GeneralApplication/db/internaldatabase.lock.db]
> > > > > > but has failed to stop it. This is very likely to create a
> > > > > > memory
> > > > > > leak.
> > > > > > 
> > > > > > Why do I get these messages?
> > > > > > As I understand these messages, H2 is still running on
> > > > > > shutdown
> > > > > > and
> > > > > > the documentation they have says that the database will
> > > > > > shutdown
> > > > > > once the last connection is closed (and the connections are
> > > > > > handled
> > > > > > by the pool i.e. Tomcat)
> > > > > > 
> > > > > > Additionally the input from H2 dev is that these messages
> > > > > > are
> > > > > > a
> > > > > > Tomcat problem. Tomcat should have first disposed of the
> > > > > > connection
> > > > > > pool and then log the running threads.Their explanation
> > > > > > seem
> > > > > > correct
> > > > > > to me.
> > > > > > 
> > > > > > Is this a known issue? Why do I see these errors in
> > > > > > catalina?
> > > > > > Should
> > > > > > I somehow dispose off the connection pool myself?
> > > > > > 
> > > > > > Thank you for your time
> > > > > > Best Regards,
> > > > > > FH
> > > > > > 
> > > > > 
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> > > > > For additional commands, e-mail: users-help@tomcat.apache.org
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > 
> > > > 
> > > > 
> > 
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> > For additional commands, e-mail: users-help@tomcat.apache.org
> 
> 
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message