tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christopher Schultz <>
Subject Re: Clustering Question...
Date Thu, 24 Sep 2009 18:59:03 GMT
Hash: SHA1


On 9/24/2009 10:21 AM, Darren Kukulka wrote:
> In a 2-node scenario, where both nodes are configured identically and
> load balanced via Apache based on availability, how can we configure the
> cluster to deal with situations where one node has exhausted its Old Gen
> heap allocation.

Hmm... this is a situations that is difficult to detect using remote
code (like mod_jk).

> In such situations we've observed that the application being served by
> the cluster slow down considerably.  I can understand why this would be
> the case for sessions on the degraded node, but why would sessions on
> the good node suffer?

Are you using session replication? If so, the "good" Tomcat may be
slowing down attempting to replicate session changes to the "damaged"
Tomcat that is either not responding, or responding slowly, or
responding in confusing ways.

> How can we modify our configuration to deal with such occurrences more
> effectively?

After we had some trouble with OOMEs in production (legit ones,
actually: we just needed more heap), I implemented a quick-and-dirty
OOME checker. All it does is "grep OutOfMemoryError catalina.out" and,
if found, sends an email to someone.

Instead of emailing, you could have your OOME checker actually shut down
(or forceably terminate) the damaged Tomcat, and then the cluster should
stabilize. With only two nodes, this might be a problem, as the good
Tomcat will take over and might, under the new load of 100% of your
traffic, experience its own pergmen exhaustion and also be shut down.

You should consider adjusting your pergmen allocation (duh!) as well as
perhaps your heap allocation as well.

- -chris
Version: GnuPG v1.4.9 (MingW32)
Comment: Using GnuPG with Mozilla -


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message