ok, so there is a leak in the Java Heap, one way to figure out what is
going would be to add
-XX:+*HeapDumpOnOutOfMemoryError
to the command line options.
This will dump the heap to a file that you can analyze. You need some
serious RAM on a box to analyze a thread dump like that, but it will
tell you everything.
I use www.yourkit.com when I analyze heapdumps
Filip
*
Dan D. wrote:
> Following is the exception that is thrown, along with a few
> surrounding lines from the catalina.out log file:
>
> [Full GC 1982271K->1982271K(1999424K), 7.5088700 secs]
> [Full GC 1982271K->1982269K(1999424K), 7.5053160 secs]
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [Full GC 1982271K->1982271K(1999424K), 7.4678950 secs]
> [Full GC 1982271K->1982246K(1999424K), 9.5055930 secs]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:288)
> at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:413)
> Caused by: java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOf(Arrays.java:2882)
> at
> java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
>
> at
> java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:390)
> at java.lang.StringBuffer.append(StringBuffer.java:224)
> at java.io.StringWriter.write(StringWriter.java:95)
> at java.io.PrintWriter.write(PrintWriter.java:412)
> at java.io.PrintWriter.write(PrintWriter.java:429)
> at java.io.PrintWriter.print(PrintWriter.java:559)
> at java.io.PrintWriter.println(PrintWriter.java:695)
> at java.lang.Throwable.printStackTrace(Throwable.java:512)
> at
> java.util.logging.SimpleFormatter.format(SimpleFormatter.java:72)
> at java.util.logging.StreamHandler.publish(StreamHandler.java:179)
> at
> java.util.logging.ConsoleHandler.publish(ConsoleHandler.java:88)
> at java.util.logging.Logger.log(Logger.java:472)
> at java.util.logging.Logger.doLog(Logger.java:494)
> at java.util.logging.Logger.logp(Logger.java:694)
> [Full GC 1982271K->1982271K(1999424K), 7.5629480 secs]
> [Full GC 1982271K->1982247K(1999424K), 7.4791920 secs]
> at org.apache.juli.logging.DirectJDKLog.log(DirectJDKLog.java:167)
> at
> org.apache.juli.logging.DirectJDKLog.error(DirectJDKLog.java:135)
> at
> org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:929)
>
> at
> org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:883)
>
> at
> org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:492)
> at
> org.apache.catalina.startup.HostConfig.start(HostConfig.java:1138)
> at
> org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:311)
>
> at
> org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:117)
>
> at
> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
> at
> org.apache.catalina.core.StandardHost.start(StandardHost.java:719)
> at
> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
> at
> org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
> at
> org.apache.catalina.core.StandardService.start(StandardService.java:516)
> at
> org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
> at org.apache.catalina.startup.Catalina.start(Catalina.java:566)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [Full GC 1982271K->1982271K(1999424K), 7.4982800 secs]
> [Full GC 1982271K->1982271K(1999424K), 7.4976100 secs]
> [Full GC 1982271K->1982244K(1999424K), 9.5026360 secs]
> [Full GC 1982272K->1982244K(1999424K), 7.4494750 secs]
> [Full GC 1982272K->1982272K(1999424K), 7.4763190 secs]
> [Full GC 1982272K->1982234K(1999424K), 9.3931880 secs]
> [Full GC[Unloading class
> com.atlassian.confluence.event.events.plugin.PluginDisableEvent]
> [Unloading class org.apache.log4j.spi.RepositorySelector]
> [Unloading class org.apache.xerces.xni.grammars.Grammar]
> [Unloading class org.apache.xerces.dom.DocumentTypeImpl]
> [Unloading class
> org.springframework.web.util.IntrospectorCleanupListener]
> [Unloading class org.apache.xerces.impl.io.UTF8Reader]
> [Unloading class org.apache.xerces.impl.dtd.XMLDTDDescription]
>
> <snip> (many more classes are unloaded here)
>
> [Unloading class org.springframework.web.context.ContextLoaderListener]
> [Unloading class com.atlassian.core.util.XMLUtils]
> [Unloading class
> org.springframework.beans.factory.support.ReplaceOverride]
> 1982271K->1982271K(1999424K), 7.4622510 secs]
> [Full GC 1982271K->1981801K(1999424K), 8.9731420 secs]
>
> And the Full GC's continue for a little while, and eventually the
> process just dies off.
>
>
> ----- Original Message ----- From: "Filip Hanik - Dev Lists"
> <devlists@hanik.com>
> To: "Tomcat Users List" <users@tomcat.apache.org>
> Sent: Monday, February 04, 2008 5:22 PM
> Subject: Re: Out of Memory Errors when upgrading from 5.0.27 to 6.0.14
>
>
>> what's the message after
>>
>> java.lang.OutOfMemoryError
>>
>> as there can be many different types of errors.
>>
>> Filip
>>
>> Dan D. wrote:
>>> Hello all,
>>>
>>> I'm currently working on upgrading our systems from using Tomcat
>>> 5.0.27 to 6.0.14.
>>>
>>> The upgrades are not being done in place (ie, we will have both
>>> systems out there, and will be doing migrations to the new
>>> version). I have been able to resolve most of the issues we have
>>> run into thus far, however, I am now somewhat stuck, and am hoping
>>> someone here may be able to help.
>>>
>>> About the environment:
>>> Shared hosting, with approximately 300-600 users on a server, with
>>> an average of ~1500 applications (many users have a few
>>> applications). Not all users are utilizing any java at all,
>>> however. Apache 1.3.33 is running and configured to serve static
>>> content.
>>>
>>> The older servers with TC5 on them have ~6GB of available RAM, new
>>> servers for TC6 have 8GB of RAM.
>>> Customer content is stored on a remote NAS server.
>>>
>>> While I understand that having the <Host> and <Context> declaration
>>> for all users listed in server.xml, that is how we have it currently
>>> configured.
>>> About the problem:
>>> When tomcat is started and running on our existing 5.0.27 server, it
>>> uses approximately 1GB of RAM, and successfully runs all
>>> applications without issue.
>>>
>>> When starting tomcat on the 6.0.14 server with the _exact_ same
>>> content, however, it runs out of heap space. This is the issue I'm
>>> running into right now that I can't seem to find any answers about.
>>> Attempting to increase the available heap results in an error
>>> allocating memory upon startup.
>>>
>>>
>>> The output from jmap seems to show that the majority of the
>>> heap/memory is being consumed by character arrays.
>>>
>>> When viewing the {tomcat.home}/work/{engine}/{application}
>>> directory, it appears there is a fair difference on how these are
>>> loaded into tomcat.
>>>
>>> On Tomcat5, this directory consists of only directories that contain
>>> a WEB-INF directory in them, and each of those have a tldCache.ser
>>> file.
>>>
>>> On Tomcat6, this directory consists of all directories located
>>> inside a directory that contains a WEB-INF directory.
>>>
>>> For example:
>>> Tomcat 6:
>>> [root@servername ]# pwd ; find .
>>> /web/tomcat/work/hosting/sitename.com
>>> .
>>> ./_
>>> ./cgi
>>> ./stats
>>> ./php_uploads
>>> ./images
>>> ./ad
>>> ./Video
>>> ./Templates
>>> ./Flv
>>> ./Songs
>>> ./newwebsite
>>> ./old files
>>> ./manager
>>>
>>> Tomcat5:
>>> [root@servername ]# pwd; find .
>>> /web/tomcat/work/hosting/sitename.com
>>> .
>>> ./_
>>> ./_/tldCache.ser
>>>
>>> And the contents of the actual home directory are:
>>> [root@servername html]# find . -maxdepth 1 -type d
>>> .
>>> ./cgi
>>> ./stats
>>> ./php_uploads
>>> ./WEB-INF
>>> ./images
>>> ./ad
>>> ./Video
>>> ./Templates
>>> ./Flv
>>> ./Songs
>>> ./newwebsite
>>> ./old files
>>>
>>> I have read http://wiki.apache.org/tomcat/FAQ/Memory and attempted
>>> all the suggestions there, to no avail.
>>>
>>> Thus, it appears that on Tomcat6, it is attempting to load all of
>>> these paths into memory, and eventually runs out. Tomcat5 did not
>>> load all of the paths to directories without applications, and thus
>>> uses less of the heap.
>>>
>>> Does it seem like I am on the right track here? Have others
>>> encountered this in the past? Is there a way to prevent this
>>> behavior within Tomcat 6?
>>>
>>> I thank you for any insight you can provide on this issue.
>>>
>>> Sincerely,
>>> Dan D.
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>>
>>> No virus found in this incoming message.
>>> Checked by AVG Free Edition. Version: 7.5.516 / Virus Database:
>>> 269.19.19/1257 - Release Date: 2/3/2008 5:49 PM
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To start a new topic, e-mail: users@tomcat.apache.org
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>>
>
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
>
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
|