tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christopher Schultz <ch...@christopherschultz.net>
Subject Re: Tomcat pausing and no java process
Date Mon, 18 Jun 2012 17:41:49 GMT
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Miguel,

(Bringing this back onto the list)

On 6/17/12 5:11 AM, Miguel González Castaños wrote:
> This weekend I made a test to upgrade from 1.5 32 bits jdk to a 1.7
> 64 bits jdk. Unfortunately even I increase the heap memory to 1gb,
> the system gets an outofmemory error after 10-15 minutes of being
> idle

Going from a 32-bit JVM to a 64-bit JVM is likely to *increase* your
webapp's memory needs even with the same heap size. The reason is that
all pointers (both native and Java references) will be double-wide
with respect to a 32-bit JVM. You can run with compressed OOPs but
that doesn't cover the JVM itself: you will necessarily require more
memory to run a 64-bit JVM.

If your webapp runs out of memory after 15 minutes of idle time, then
something is terribly wrong. I would take the following steps:

1. Except for -Xmx (and maybe -Xms), remove *all* heap-related settings
   from your JVM launch command. Your existing settings look like
   failed attempts to solve a memory leak within the webapp.

2. Launch your webapp in a safe environment (you *do* have an
   environment identical to production where you can to testing like
   this, right?) and let it sit idle. Take a thread dump and see
   if any threads exist that you aren't expecting: that is, something
   other than a bunch of idle request-processor threads. Maybe your
   webapp performs some kind of background operation periodically,
   and that's okay too. Anything you don't recognize: investigate.

3. Take heap dumps at intervals or maybe 5 minutes (that should give
   you 3 heap dumps before your "idle" webapp fails and the JVM
   dies). Load-up those heap-dumps in a heap-analysis tool and
   look for differences. Maybe you'll find out that you have a million
   foo.bar.Cache objects being created every minute or something.
   Obviously investigate those.

4. Once you can get your webapp to be stable during /idle/ time, run a
   load-test against it (you *do* have a load-test for your webapp,
   right?) and repeat step #3 above.

Using these steps, you ought to be able to identify and fix 90% of
memory leaks with very little trouble.

> Now I have installed a 1.7 32 bits jdk and leave the heap memory to
> what it was with 1.5 32 bits jdk:  Xms128m and Xmx512m. It seems to
> be stable.
> 
> Unfortunately this heap size proofed last week not being enough. I 
> analyzed the heap dumps and I have realized there is a process that
> eats up 200-300 mb. This thread what it does is to load a 5k-10k
> objects in memory that represent an email and other info that is
> used to send emails to our clients. From my perspective of an old
> coder that tried to use memory as less as I could loading such big
> number of objects to send 200-300 emails an hour doesn't scale.
> Specially if this company expects to grow in the future and send
> more emails.

Agreed. You might want to separate this email thing into a separate
process, so that if/when it takes up a lot of memory and dies, it
doesn't bring-down your webapp at the same time.

> As I see it, increasing the heap is not the solution, but it can be
> a patch until we get another programmer on board and can refactor
> the code. Also the owners want to have the impression that the
> webapp is stable again and feel secure to perform next steps: get a
> bigger dedicated server (not a virtual one) and get a new
> programmer/company to do things more professionally.

Good plan. Otherwise, you will not have a stable product. Investing in
identifying and fixing whatever is wrong is money well-spent.

> So the question is, how can I increase the heap? Any log that give
> me any clue? I have separated gc.log to see how the young, perm and
> old generations numbers evolve.

You want to increase the heap? You just need to use -Xmx. You can use
jconsole or even jmap to ask the JVM what the effective heap size is
once it has been launched.

- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk/faF0ACgkQ9CaO5/Lv0PCM/wCgmYWZb47/twSvlgPn6d2y4Dlq
rs0AoK7ouint5uVkx3kaFsRZ8W68uBeK
=qSCj
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message