tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Hansen <joe.hansen...@gmail.com>
Subject Re: java.lang.OutOfMemoryError: Java heap space
Date Tue, 06 Oct 2009 16:23:32 GMT
Rainer,

Thanks for looking at those long thread dumps for me!!

I am sorry. I did NOT take these dumps at the right time (i.e. when
Tomcat was inundated with requests and couldn't cope with the load).
After I increased the heap size to 512MB (from 64MB default), I am not
getting the OutOfMemoryError(s) anymore. After I set KeepAlive On
(Thanks Andre!), the number httpd processes isn't increasing either.
The number of httpd processes increased from 8 to 21 and stayed there
for more than 16 hours now. If the number of httpd processes gets out
of control again, I will definitely take thread dumps once again.

However, I doubt the issue is fixed because I should have seen this
issue long time ago (since I haven't changed any code nor the traffic
to our websites increased in a long while).

Should I just wait and see or are there any tests that I can do?

Your contribution to this forum is amazing, Rainer. I am grateful to
you and Andre for your efforts. Thank you!

Regards,
Joe



On Tue, Oct 6, 2009 at 7:25 AM, Rainer Jung <rainer.jung@kippdata.de> wrote:
> On 05.10.2009 18:58, Joe Hansen wrote:
>> Thank you so much for your tips, Rainer!
>>
>> The websites went down yet again. Increasing the java heap size took
>> care of the OutOfMemoryError, but the number of httpd processes keep
>> increasing until the websites crash. I haven't added any new code in
>> the past few months, hence I am surprised why the requests are getting
>> stuck. Here's a link to the tomcat thread dumps:
>> http://pastebin.com/m17eea139
>>
>> Please let me know if you cannot view it and I will email the relevant
>> portion of the catalina.out file to you. Is there an easy way to find
>> out what code is causing the requests to get stuck?
>
> The dump file contains three thread dumps.
>
> The things all dumps have in common:
>
> - 60 threads for the quartz scheduler, all idle
> - 13 threads in the AJP connection pool, connected to Apache, but idle
> waiting for the next request to be send (the same threads in all three
> dumps)
> - 6 store plus 6 expiry threads of the EHCache, seems idle
> - 1 AJP + 1 HTTP(S) thread (port 8443) waiting to accept the next new
> connection to come in
> - 2 AJP + 3 HTTP(S) threads (port 8443) sitting idle the pool, waiting
> for work
> - a couple of other normal threads not directly related to request handling
>
> So the time you took the three dumps, this Tomcat was completely idle
> and did not have a single request to handle.
>
> If you are completely sure, you took the dumps while there was a storm
> of requests and your system couldn't cope the load, something has
> prevented the requests to ever reach Tomcat.
>
> I don't have your Tomcat version at hand at the moment, but for some
> time very special OutOfMemory errors (could not create native thread)
> lead to a situation, where Tomcat simply wouldn't accept any new
> connections. Although you report OutOfMemory errors, I'm not directly
> suggesting that that is your problem here. There might still be a
> relation though.
>
> Are you sure, that you took the dumps for the right Tomcat at the right
> time?
>
> Regards,
>
> Rainer
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message