httpd-bugs mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bugzi...@apache.org
Subject DO NOT REPLY [Bug 48024] mod_sed consumes too much memory when used inside reverse proxy and line sizes are very long e.g. 1-5M
Date Mon, 19 Oct 2009 18:54:07 GMT
https://issues.apache.org/bugzilla/show_bug.cgi?id=48024

--- Comment #1 from Basant Kumar Kukreja <basant.kukreja@sun.com> 2009-10-19 11:54:06
UTC ---
Created an attachment (id=24397)
 --> (https://issues.apache.org/bugzilla/attachment.cgi?id=24397)
Patch

sed needs to buffer the line until it finds the end of the line. 
At present it increments the line buffer by 4KB size or based on new sizes.
When mod_sed is used inside apache with regular files, number of time
reallocation happens are less because single brigade contains the entire
file.

But when mod_sed is used as a reverse proxy then the number of times
sed_response is called is called can be huge which means buffer will be
reallocated many many times compared to filtering text directly.  I noticed
that for 2M characters in single line then mod_sed reallocated stuff in 275
times.

Fix for this is that grow_buffer should grow the sizes at least each time it
is invoked. This reduced the the reallocation to just 9 times for 2M line
size.

Fix is attached.

Bram confirmed that fix worked for him. Here is the comments from Bram.
----------------------------------
For a 2 MB, 3 MB and 4 MB file it seems to be below 26/27 MB of memory.
For a 20 MB file it is around 98 MB of memory
----------------------------------

-- 
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


Mime
View raw message