httpd-bugs mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bugzi...@apache.org
Subject DO NOT REPLY [Bug 44123] New: - Apache slowly consumes all system memory while running streaming CGI script.
Date Fri, 21 Dec 2007 22:28:29 GMT
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG·
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
<http://issues.apache.org/bugzilla/show_bug.cgi?id=44123>.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND·
INSERTED IN THE BUG DATABASE.

http://issues.apache.org/bugzilla/show_bug.cgi?id=44123

           Summary: Apache slowly consumes all system memory while running
                    streaming CGI script.
           Product: Apache httpd-2
           Version: 2.2.3
          Platform: Other
        OS/Version: Linux
            Status: NEW
          Severity: major
          Priority: P2
         Component: mod_cgi
        AssignedTo: bugs@httpd.apache.org
        ReportedBy: garrett.gallant@ge.com


Hello, I'm trying to solve a problem we're having where some of our long-
running streaming CGI scripts are causing Apache to grow continuously, 
eventually consuming all system memory and starving Apache and other processes 
(manifested as Apache returning HTTP 500 (internal server error) responses to 
subsequent requests to other resources, and other weirdness). I've reduced the 
test to a really simple CGI script which reproduces the memory growth reliably 
for me (see steps to reproduce below).

I'm wondering if this is normal behaviour (is continuous streaming not 
supported?), or perhaps I'm just mangling my headers or missing something 
really obvious. Any insight you guys have would be great. If it's a bug in 
Apache, my apologies for darkening your door at Christmastime. ;-)

I've observed the behaviour on both of our setups:

Linux: 2.6.13.4.D400-LEAF (a custom (i.e. suspect) roll-our-own Linux - gcc 
version 4.0.2 20051125, Red Hat 4.0.2-8).
Apache: 2.0.55

Linux: Ubuntu Fiesty (standard, nothing special)
Apache: 2.2.3

Steps to reproduce:
-------------------
1. Restart Apache.
2. Point a browser at the CGI script (see offending cgi script (1) below) 
running either runPlain() or runMultipart().
3. Run ps every 10 seconds for 10 minutes (see ps log script (2) below).

My observations:
-----------------
1. In both the plain/text and multipart case, I observed Apache's memory 
growing linearly at a rate of ~19.6kB/min (typically in 4kB increments 
depending on your Linux/MMU page size).

2. When I stopped the script after 5 minutes, Apache did not appear to free its 
memory. Then, when I restarted the script, Apache ran flat for 5 minutes before 
starting to grab more memory. This suggests that when a CGI script stops, 
Apache frees some of its internal memory pool, but does not release it to 
system memory, instead reusing it for future CGI scripts. I found I had to 
restart Apache after each test to avoid masking the memory growth of subsequent 
tests.

3. I tried various data rates in my below scripts, by adjusting the sleep time, 
chars per line and lines per cycle, and found the memory growth rate was 
roughly a linear function of the CGI's data output rate (with the function 
flattening at high data rates probably due to CPU/network saturation).

4. I tried various other stabs in the dark, like flushing stdout on each cycle 
(before the sleep). No change to the growth rate.

1. Offending cgi script:
------------------------
#include <time.h>
#include <dirent.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/time.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <unistd.h>

/*
main.
*/
int main(int argc, char *argv[])
{
    runPlain();
}

/*
Run a text/plain test.
*/
int runPlain()
{
    int i,j;
    printf("Content-type: text/plain; charset=us-ascii\n\n");
    for (i=0; ; i++)
    {
        for (j=0; j<5; j++)
        {
            printf("%d,%d: wakka wakka wakka wakka wakka wakka wakka\n",i,j);
        }
        printf("\n");
        usleep(5000);
    }
}

/*
Run a multipart/mixed-replace test
per http://www.w3.org/Protocols/rfc1341/7_2_Multipart.html section 7.2.1.
*/
int runMultipart()
{
    int i,j,num;
    printf("Content-type: multipart/mixed-replace; boundary=myboundary\n\n");
    printf("Preamble\n");
    for (i=0; ; i++)
    {
        printf("--myboundary\n");
        printf("Content-type: text/plain; charset=us-ascii\n\n");
        for (j=0; j<5; j++)
        {
            printf("%d,%d: wakka wakka wakka wakka wakka wakka wakka\n",i,j);
        }
        printf("\n");
        usleep(5000);
    }
}

2. ps log:
----------
#!/bin/sh

interval=$1
while :
do
  timestamp=`date +%Y.%m.%d-%H.%M.%S`
  ps -o pid,pcpu,mem,cmd | sed "s/^/$timestamp  /"
  sleep $interval
done

-- 
Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


Mime
View raw message