tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zoran Avtarovski <zo...@sparecreative.com>
Subject Re: Help in diagnosing server unresponsiveness
Date Sun, 03 Feb 2013 03:44:38 GMT
I'm using the Sun JVM 1.6.0_31-b04.

We had other performance issues with openJDK.

Z.

On 3/02/13 2:41 PM, "Edson Richter" <edsonrichter@hotmail.com> wrote:

>Em 03/02/2013 01:35, Zoran Avtarovski escreveu:
>> Thanks for the advice.
>>
>> All libraries are within the apps WEB-INF folder.
>>
>> It also doesn't appear to be a memory leak. Typically I would expect
>> memory use to increase over time with a memory leak, but our app shows
>>no
>> increase just a sharp spike to max allocated memory prior to becoming
>> unresponsive.
>
>Which VM are you using?
>I had similar problems (even Kernel Panic!) running OpenJDK (default in
>some Linux distros).
>
>Regards,
>
>Edson
>
>>
>> For persistence we use Mybatis ORM which we have no issues with on other
>> apps.
>>
>> I've set the GC logging up, but because there are no OOME no thread
>>dumps
>> are generated.
>>
>> Z.
>>
>> On 2/02/13 4:09 AM, "Edson Richter" <edsonrichter@hotmail.com> wrote:
>>
>>> Em 01/02/2013 15:03, Edson Richter escreveu:
>>>> Removing the hardware issues (faulty memory or disk), that you
>>>> obviously already tested, I'll try to give some directions for
>>>>testing:
>>>>
>>>> a) Main cause of memory leaks are hard references in main class
>>>> loader. This happens when you put all your libraries into
>>>> $TOMCAT_HOME/lib. Try to move your libraries into WEB-INF/lib and
>>>> check how does your app behave.
>>>> b) Lots of people forget to correctly close external resources (files,
>>>> tcp connections, jdbc resources). Check your source code using
>>>> FindBugs. It is not perfect, but will give you lots of warnings if you
>>>> run on risk of not correctly closing resources. Remember, for jdbc
>>>> resources, you should close all result sets first, then all
>>>> statements, then all connections (not all database drivers will
>>>> release resultset resources on statement close!).
>>>> c) Also, we see incorrect thread programming...
>>> When I say (we see..) I want to mean: I see lots of junior programmers
>>> doing the same mistake over and over... I don't know if this is your
>>> case, but I feel that worth to mention here.
>>>
>>>
>>>> remember to have a way to signalize to your threads that the
>>>> application is closing or reloading. In my apps, I have a LifeCicly
>>>> listener that will notify all threads that they should shutdown
>>>> immediately. If a thread is stuck using resources, it will remain with
>>>> that forever...
>>>> d) If your app is JDK6 compatible, give a try on JRockit VM (from
>>>> Oracle), and the excellent JRockit Mission Control that helps you to
>>>> identify problems in real time.
>>>> e) Never store content in static classes. The references stay forever.
>>>> If you are using JPA, let JPA implementation to handle the cache. Use
>>>> Soft Weak or Hard Weak if using EclipseLink.
>>>> f) Never ever use OneToMany just because it is easy. If you have one
>>>> object that has OneToMany to other 100, that these has One to many to
>>>> another 100, you will have 10001 objects in memory with 1 query that
>>>> is supposed to returns 1 record (the first object). If your query
>>>> returns 10 records of the first object, you would have 100001 object
>>>> in memory... If you have 20 users using different objects... well, you
>>>> got the point, right?
>>>>
>>>> By using good server hardware (ECC memory, SCSI disks, etc), a stable
>>>> linux distro (my preference is for CentOS 64bit), and following the
>>>> rules above, I manage to have web apps that run withing 2Gb of memory
>>>> (on 8Gb of hardware), hundred of users in databases with > 20Gb, 24x7.
>>>>
>>>>
>>>> I hope this helps,
>>>>
>>>> Edson Richter
>>>>
>>>>
>>>> Em 31/01/2013 23:36, Zoran Avtarovski escreveu:
>>>>> Hi Guys,
>>>>>
>>>>> We have a application running on the latest Tomcat7 and we are
>>>>>getting
>>>>> a
>>>>> server crash or becoming unresponsive. This occur every few days at
>>>>> no fixed
>>>>> intervals or time of day and they certainly don't correlate to any
>>>>>app
>>>>> function ­ at least not according to the logs.
>>>>>
>>>>> We set setup monitoring using JavaMelody and what we see is dramatic
>>>>> spikes
>>>>> in CPU and memory usage at the time of the crash.
>>>>>
>>>>> Memory hovers around 3-5% for the rest of the time and CPU is the
>>>>>same.
>>>>>
>>>>> I've looked at the number of sessions, HTTP activity , jdbc activity
>>>>> and
>>>>> nothing obvious jumps out.
>>>>>
>>>>> I'd really appreciate your collective wisdom in putting into practice
>>>>> some
>>>>> strategies to identify the cause of the spikes. This driving me and
>>>>> my team
>>>>> nuts.
>>>>>
>>>>> Any help would be appreciated.
>>>>>
>>>>> Z.
>>>>>
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>>
>>
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>For additional commands, e-mail: users-help@tomcat.apache.org
>



---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message