tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christopher Schultz <>
Subject Re: Notification strategy for OutOfMemoryError
Date Wed, 11 Dec 2013 15:14:58 GMT
Hash: SHA256


On 12/9/13, 8:20 PM, Bill Davidson wrote:
> On 12/9/2013 3:12 PM, Christopher Schultz wrote:
>> Was it a transient error, or a chronic condition? A single thread
>> can, for instance, spew objects into its stack or eden space
>> exhausting memory but, when that thread hits the OOME, all those
>> objects are freed which basically recovers from the situation.
>> If, instead, you fill-up some shared cache, buffer, etc. and NO 
>> threads can get more memory, then you're basically toast.
>> Which of the above was it?
> It looked more like the first one though we still haven't tracked
> down the cause.  We had several dozen threads running at the time.
> That's common for us.  It's not that unusual for us to have a
> couple of hundred users with active sessions per server at any
> given time.
>> There are a bunch of things you can try to do. They all have
>> their caveats, failure scenarios, and inefficacies.
>> 1. Use -XX:OnOutOfMemoryError="cmd args;cmd args"
>> Rig this to email you, register a passive-check data point with
>> your monitoring server, etc. Just remember that OOMEs happen for
>> a number of reasons. You could have run out of file handles or
>> you could have run out of heap space.
> That looks interesting.  It wouldn't tell me about the error but
> at least I'd know that there was an OOME.  Better than nothing and
> I can go check catalina.out.  Of course, I still have the problem
> that threads silently fail and show my users not so much as an
> error message.
>> 2. Use JMX monitoring, set java.lang:MemoryPool/[heap 
>> space]/UsageThreshold to whatever byte value you want to set as
>> your limit. Then, check java.lang:MemoryPool/[heap 
>> space]/UsageThresholdExceeded to see if it is true. If so, your
>> usage threshold has been exceeded.
>> Note that this is not proof-positive than an OOME occurred. It's
>> also tough to tell what value to use for the threshold. You can't
>> really set it to MaxHeap - 1 byte, because you'll never get that
>> value in practice. If you set it too low, you'll get warnings all
>> the time when your heap usage rises in the normal course of
>> business.
> I'm less enthused about that one.
>> 3. catch IOException in a filter and set an application
>> attribute. Check this attribute from your monitor.
>> I've been considering doing this, because I can rig it so that
>> the error handler does not actually require any memory to run.
>> The problem is that sometimes OOMEs interrupt one thread and not
>> another. You may not catch the OOME in that thread -- it may
>> happen in a background thread that does not go through the
>> filter.
> I'm not sure I understand this one.  How does an IOException relate
> to an OOME?

Sorry, I meant of course OutOfMemoryError. Just make sure you use as
little memory as possible during the exception handler or it will fail
itself and possibly mask the original problem.

- -chris
Version: GnuPG v1.4.15 (Darwin)
Comment: GPGTools -
Comment: Using GnuPG with Thunderbird -


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message