tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Edson Richter <edsonrich...@hotmail.com>
Subject Re: Help in diagnosing server unresponsiveness
Date Sun, 03 Feb 2013 03:41:51 GMT
Em 03/02/2013 01:35, Zoran Avtarovski escreveu:
> Thanks for the advice.
>
> All libraries are within the apps WEB-INF folder.
>
> It also doesn't appear to be a memory leak. Typically I would expect
> memory use to increase over time with a memory leak, but our app shows no
> increase just a sharp spike to max allocated memory prior to becoming
> unresponsive.

Which VM are you using?
I had similar problems (even Kernel Panic!) running OpenJDK (default in 
some Linux distros).

Regards,

Edson

>
> For persistence we use Mybatis ORM which we have no issues with on other
> apps.
>
> I've set the GC logging up, but because there are no OOME no thread dumps
> are generated.
>
> Z.
>
> On 2/02/13 4:09 AM, "Edson Richter" <edsonrichter@hotmail.com> wrote:
>
>> Em 01/02/2013 15:03, Edson Richter escreveu:
>>> Removing the hardware issues (faulty memory or disk), that you
>>> obviously already tested, I'll try to give some directions for testing:
>>>
>>> a) Main cause of memory leaks are hard references in main class
>>> loader. This happens when you put all your libraries into
>>> $TOMCAT_HOME/lib. Try to move your libraries into WEB-INF/lib and
>>> check how does your app behave.
>>> b) Lots of people forget to correctly close external resources (files,
>>> tcp connections, jdbc resources). Check your source code using
>>> FindBugs. It is not perfect, but will give you lots of warnings if you
>>> run on risk of not correctly closing resources. Remember, for jdbc
>>> resources, you should close all result sets first, then all
>>> statements, then all connections (not all database drivers will
>>> release resultset resources on statement close!).
>>> c) Also, we see incorrect thread programming...
>> When I say (we see..) I want to mean: I see lots of junior programmers
>> doing the same mistake over and over... I don't know if this is your
>> case, but I feel that worth to mention here.
>>
>>
>>> remember to have a way to signalize to your threads that the
>>> application is closing or reloading. In my apps, I have a LifeCicly
>>> listener that will notify all threads that they should shutdown
>>> immediately. If a thread is stuck using resources, it will remain with
>>> that forever...
>>> d) If your app is JDK6 compatible, give a try on JRockit VM (from
>>> Oracle), and the excellent JRockit Mission Control that helps you to
>>> identify problems in real time.
>>> e) Never store content in static classes. The references stay forever.
>>> If you are using JPA, let JPA implementation to handle the cache. Use
>>> Soft Weak or Hard Weak if using EclipseLink.
>>> f) Never ever use OneToMany just because it is easy. If you have one
>>> object that has OneToMany to other 100, that these has One to many to
>>> another 100, you will have 10001 objects in memory with 1 query that
>>> is supposed to returns 1 record (the first object). If your query
>>> returns 10 records of the first object, you would have 100001 object
>>> in memory... If you have 20 users using different objects... well, you
>>> got the point, right?
>>>
>>> By using good server hardware (ECC memory, SCSI disks, etc), a stable
>>> linux distro (my preference is for CentOS 64bit), and following the
>>> rules above, I manage to have web apps that run withing 2Gb of memory
>>> (on 8Gb of hardware), hundred of users in databases with > 20Gb, 24x7.
>>>
>>>
>>> I hope this helps,
>>>
>>> Edson Richter
>>>
>>>
>>> Em 31/01/2013 23:36, Zoran Avtarovski escreveu:
>>>> Hi Guys,
>>>>
>>>> We have a application running on the latest Tomcat7 and we are getting
>>>> a
>>>> server crash or becoming unresponsive. This occur every few days at
>>>> no fixed
>>>> intervals or time of day and they certainly don't correlate to any app
>>>> function ­ at least not according to the logs.
>>>>
>>>> We set setup monitoring using JavaMelody and what we see is dramatic
>>>> spikes
>>>> in CPU and memory usage at the time of the crash.
>>>>
>>>> Memory hovers around 3-5% for the rest of the time and CPU is the same.
>>>>
>>>> I've looked at the number of sessions, HTTP activity , jdbc activity
>>>> and
>>>> nothing obvious jumps out.
>>>>
>>>> I'd really appreciate your collective wisdom in putting into practice
>>>> some
>>>> strategies to identify the cause of the spikes. This driving me and
>>>> my team
>>>> nuts.
>>>>
>>>> Any help would be appreciated.
>>>>
>>>> Z.
>>>>
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message