httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stas Bekman <s...@stason.org>
Subject Re: filtering huge request bodies (like 650MB files)
Date Thu, 11 Dec 2003 08:54:20 GMT
Sander Striker wrote:
> On Thu, 2003-12-11 at 07:44, Stas Bekman wrote:
> 
> 
>>I now know what the problem is. It is not a problem in httpd or its filters, 
>>but mod_perl, allocated filter struct from the pool. With many bucket brigades 
>>there were many filter invocations during the same request, resulting in 
>>multiple memory allocation. So I have to move to the good-old malloc/free to 
>>solve this problem.
>>
>>Though it looks like I've found a problem in apr_pcalloc.
>>
>>modperl_filter_t *filter = apr_pcalloc(p, sizeof(*filter));
>>
>>was constantly allocating 40k of memory, whereas sizeof(*filter) == 16464
>>
>>replacing apr_pcalloc with apr_palloc reduced the memory allocations to 16k.
>>
>>Could it be a bug in APR_ALIGN_DEFAULT? apr_pcalloc calls APR_ALIGN_DEFAULT 
>>and then it calls apr_palloc which calls APR_ALIGN_DEFAULT again, and probably 
>>doubling the memory usage.
> 
> 
> Woah!  I'll look into this.

I think pcalloc shouldn't call the alignment function, but let palloc align 
the size and return the updated size to pcalloc, so it could memset the right 
size.

On the other hand, the "right" size is the one requested by the caller, so the 
caller really expects to zero only the amount it has requested. So may be it's 
OK if palloc allocates more memory (because of the alignment), but calloc 
zeroing only the requested amount. In which case all we need to do is nuke the 
call to APR_ALIGN_DEFAULT in pcalloc.

__________________________________________________________________
Stas Bekman            JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/     mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org   http://ticketmaster.com


Mime
View raw message