tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zoran Avtarovski <zo...@sparecreative.com>
Subject Re: Help in diagnosing server unresponsiveness
Date Sun, 03 Feb 2013 03:35:41 GMT
Thanks for the advice.

All libraries are within the apps WEB-INF folder.

It also doesn't appear to be a memory leak. Typically I would expect
memory use to increase over time with a memory leak, but our app shows no
increase just a sharp spike to max allocated memory prior to becoming
unresponsive.

For persistence we use Mybatis ORM which we have no issues with on other
apps.

I've set the GC logging up, but because there are no OOME no thread dumps
are generated.

Z.

On 2/02/13 4:09 AM, "Edson Richter" <edsonrichter@hotmail.com> wrote:

>Em 01/02/2013 15:03, Edson Richter escreveu:
>> Removing the hardware issues (faulty memory or disk), that you
>> obviously already tested, I'll try to give some directions for testing:
>>
>> a) Main cause of memory leaks are hard references in main class
>> loader. This happens when you put all your libraries into
>> $TOMCAT_HOME/lib. Try to move your libraries into WEB-INF/lib and
>> check how does your app behave.
>> b) Lots of people forget to correctly close external resources (files,
>> tcp connections, jdbc resources). Check your source code using
>> FindBugs. It is not perfect, but will give you lots of warnings if you
>> run on risk of not correctly closing resources. Remember, for jdbc
>> resources, you should close all result sets first, then all
>> statements, then all connections (not all database drivers will
>> release resultset resources on statement close!).
>> c) Also, we see incorrect thread programming...
>
>When I say (we see..) I want to mean: I see lots of junior programmers
>doing the same mistake over and over... I don't know if this is your
>case, but I feel that worth to mention here.
>
>
>> remember to have a way to signalize to your threads that the
>> application is closing or reloading. In my apps, I have a LifeCicly
>> listener that will notify all threads that they should shutdown
>> immediately. If a thread is stuck using resources, it will remain with
>> that forever...
>> d) If your app is JDK6 compatible, give a try on JRockit VM (from
>> Oracle), and the excellent JRockit Mission Control that helps you to
>> identify problems in real time.
>> e) Never store content in static classes. The references stay forever.
>> If you are using JPA, let JPA implementation to handle the cache. Use
>> Soft Weak or Hard Weak if using EclipseLink.
>> f) Never ever use OneToMany just because it is easy. If you have one
>> object that has OneToMany to other 100, that these has One to many to
>> another 100, you will have 10001 objects in memory with 1 query that
>> is supposed to returns 1 record (the first object). If your query
>> returns 10 records of the first object, you would have 100001 object
>> in memory... If you have 20 users using different objects... well, you
>> got the point, right?
>>
>> By using good server hardware (ECC memory, SCSI disks, etc), a stable
>> linux distro (my preference is for CentOS 64bit), and following the
>> rules above, I manage to have web apps that run withing 2Gb of memory
>> (on 8Gb of hardware), hundred of users in databases with > 20Gb, 24x7.
>>
>>
>> I hope this helps,
>>
>> Edson Richter
>>
>>
>> Em 31/01/2013 23:36, Zoran Avtarovski escreveu:
>>> Hi Guys,
>>>
>>> We have a application running on the latest Tomcat7 and we are getting
>>>a
>>> server crash or becoming unresponsive. This occur every few days at
>>> no fixed
>>> intervals or time of day and they certainly don't correlate to any app
>>> function ­ at least not according to the logs.
>>>
>>> We set setup monitoring using JavaMelody and what we see is dramatic
>>> spikes
>>> in CPU and memory usage at the time of the crash.
>>>
>>> Memory hovers around 3-5% for the rest of the time and CPU is the same.
>>>
>>> I've looked at the number of sessions, HTTP activity , jdbc activity
>>>and
>>> nothing obvious jumps out.
>>>
>>> I'd really appreciate your collective wisdom in putting into practice
>>> some
>>> strategies to identify the cause of the spikes. This driving me and
>>> my team
>>> nuts.
>>>
>>> Any help would be appreciated.
>>>
>>> Z.
>>>
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>>
>>
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>For additional commands, e-mail: users-help@tomcat.apache.org
>



---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message