perl-modperl mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From André Warnier>
Subject Re: avoiding child death by size limit
Date Fri, 11 Dec 2009 20:54:55 GMT
E R wrote:
> Hi,
> I have a problem where a mod_perl handler will allocate a lot of
> memory when processing a request, and this causes Apache to kill the
> child due to exceeding the configure child size limit.

Chances are that a child does not exceed this memory right away, at the 
first request.  More likely, it uses more and more memory at each 
request it processes, and finally after a number of requests, it exceeds 
the maximum memory and gets killed.
In other words, it is leaking.

So, paraphrasing someone else : don't treat the symptom, treat the cause.

> However, the memory allocated will get freed up or re-used by the next
> request - 

If it is a real leak, then no, it will not.

I think the memory is just fragmented enough to be
> automatically reclaimed by the memory allocator (I've heard that some
> mallocs can return memory to the OS in 1 MB chunks.)

See William's answer : unlikely.

> Are there any special techniques people use to avoid this situation?
> Does SizeLimit count actual memory used or does it just look at the
> process size?

In a previous similar exercise, in despair I used the module Devel::Leak 
as follows :

use Devel::Leak;
my $DEBUGMem = 1;
my ($SVTable,$prevSVCount,$lastSVCount);

if ($DEBUGMem) {
   $prevSVCount = Devel::Leak::NoteSV($SVTable);
   warn "[$$] before something, total SVs : $prevSVCount";

do_something(); # .. which could be leaking

if ($DEBUGMem) {
   $lastSVCount = Devel::Leak::CheckSV($SVTable);
   warn "[$$] after something : total SVs : $lastSVCount";
   warn "[$$]   new SVs : ",($lastSVCount - $prevSVCount);

It doesn't require any specially-compiled perl.
It does not actually print the memory size used.  It just provides a 
count of new "things" that have been allocated and not freed by 
do_something(). It is very rough, but it was very helpful to me to find 
out what exact piece of code was leaking "things", which is basically an 
alias for memory.
The point is, if it keeps on growing around the same piece of code each 
time you process a request, then you at least know where bad things happen.

If it happens in your code, then when you know where, it should be 
possible to fix it.  If it happens in someone else's module that you are 
using, there are usually several alternative modules for just about 
anything on CPAN.  If there aren't and you cannot do without, /then/ 
maybe you should considering limiting the number of requests that each 
child handles before it gets killed. But that should not be the first 
choice, because it is the least efficient.

View raw message