tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Asim Alp <dev.l...@educationalnetworks.net>
Subject Registry problems followed by OutOfMemory errors
Date Fri, 10 Dec 2004 15:58:41 GMT
Here is our configuration:

Windows Server 2003
Apache 2.0.49 (Win32) mod_ssl/2.0.51 OpenSSL/0.9.7d DAV/2 mod_jk/1.2.6
Tomcat 5.5.4
sun jdk 1.5.0-b64

We have a 2 node tomcat cluster each running with the -Xms128m
-Xmx1024m options.  We have the following thread options set in the
AJP/1.3 connector:
maxThreads="1000" minSpareThreads="50" maxSpareThreads="300"

And the following corresponding values in our workers.properties (same
for each tomcat)
worker.tomcat1.type=ajp13
worker.tomcat1.cachesize=300
worker.tomcat1.cache_timeout=60
worker.worker1.local_worker=1
worker.tomcat1.lbfactor=1
worker.tomcat1.connect_timeout = 1000
worker.tomcat1.prepost_timeout = 1000
worker.tomcat1.reply_timeout = 7000
...
worker.loadbalancer.local_worker_only=0

We have a heavily loaded database application running.  Same
application runs on about 180 different virtual hosts on each Tomcat.

We did profiling with JProfile and couldn't find any memory leaks in
our application.  Each Tomcat works perfect for about 8 to 10 hours,
then all of a sudden, they start hanging (not necessarily at the same
time).  We monitor our heap memory very closely and we usually have
enough FREE memory (more than 25%) when the following errors occur:

First, we get a couple of SEVERE registering errors:

<record>
  <date>2004-12-09T18:19:51</date>
  <millis>1102634391333</millis>
  <sequence>270</sequence>
  <logger>org.apache.commons.modeler.Registry</logger>
  <level>SEVERE</level>
  <class>org.apache.commons.modeler.Registry</class>
  <method>registerComponent</method>
  <thread>44</thread>
  <message>Error registering
Catalina:type=RequestProcessor,worker=jk-8009,name=JkRequest2291</ 
message>
  <exception>
    <message>javax.management.InstanceAlreadyExistsException:
Catalina:type=RequestProcessor,worker=jk-8009,name=JkRequest2291</ 
message>
    <frame>
      <class>com.sun.jmx.mbeanserver.RepositorySupport</class>
      <method>addMBean</method>
      <line>452</line>
    </frame>
    <frame>
       
<class>com.sun.jmx.interceptor.DefaultMBeanServerInterceptor</class>
      <method>internal_addObject</method>
      <line>1410</line>
    </frame>
    ....
  </exception>
</record>
<record>
  <date>2004-12-09T18:19:51</date>
  <millis>1102634391333</millis>
  <sequence>271</sequence>
  <logger>org.apache.jk.common.ChannelSocket</logger>
  <level>WARNING</level>
  <class>org.apache.jk.common.ChannelSocket</class>
  <method>registerRequest</method>
  <thread>44</thread>
  <message>Error registering request</message>
</record>

Followed by a couple of java.lang.OutOfMemoryError: PermGen space  
messages
<record>
  <date>2004-12-09T21:48:25</date>
  <millis>1102646905849</millis>
  <sequence>294</sequence>
  <logger>StandardWrapper[/apps:jsp]</logger>
  <level>SEVERE</level>
  <class>org.apache.catalina.core.StandardWrapperValve</class>
  <method>invoke</method>
  <thread>47</thread>
  <message>Servlet.service() for servlet jsp threw exception</message>
  <exception>
    <message>java.lang.OutOfMemoryError: PermGen space</message>
  </exception>
</record>
...

Followed by HUNDREDS (basically for each thread) of Error
unregistering mbean messages
<record>
  <date>2004-12-09T21:53:29</date>
  <millis>1102647209630</millis>
  <sequence>302</sequence>
  <logger>org.apache.commons.modeler.Registry</logger>
  <level>SEVERE</level>
  <class>org.apache.commons.modeler.Registry</class>
  <method>unregisterComponent</method>
  <thread>20</thread>
  <message>Error unregistering mbean </message>
  <exception>
    <message>javax.management.RuntimeOperationsException: Object name
cannot be null</message>
    <frame>
       
<class>com.sun.jmx.interceptor.DefaultMBeanServerInterceptor</class>
      <method>isRegistered</method>
      <line>545</line>
    </frame>
    <frame>
      <class>com.sun.jmx.mbeanserver.JmxMBeanServer</class>
      <method>isRegistered</method>
      <line>619</line>
    </frame>
    <frame>
      <class>org.apache.commons.modeler.Registry</class>
      <method>unregisterComponent</method>
      <line>642</line>
    </frame>
    ...
  </exception>
</record>

Any ideas?  Is this something to do with our thread counts
(maxThreads="1000" minSpareThreads="50" maxSpareThreads="300")?  If
so, how can we determine these numbers for fastest performance.  We
get about 5 hits every second and we want our Tomcats to serve static
files as well (such as image files), so we want to make sure that we
have enough threads.  It's very important for our pages to load fast
on the client side.

Thank you very much!  Any help would be greatly appreciated!

Asim


---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-user-help@jakarta.apache.org


Mime
View raw message