Date: Mon, 19 Jun 1995 13:53:58 -0700 (PDT)
From: Brian Behlendorf <brian@organic.com>
Subject: more apache 0.7.2 death notes
To: new-httpd@hyperreal.com
Message-ID: <Pine.3.89.9506191346.H28468-0100000@eat.organic.com>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Sender: owner-new-httpd@apache.org
Precedence: bulk
Reply-To: new-httpd@apache.org


Came in this morning to find hyperreal's 0.7.2h dead:

1) only the parent was still around
2) it was sucking down CPU as fast as possible
3) A kill -HUP didn't restart it
4) ktrace didn't report any system calls being made
5) no CGI scripts running that I could tell
6) the server had stopped serving up requests about 9:30am our time
7) first errant 403 request came at 8:53am, shortly after a 500 server 
error for the home page
8) looks like the "403" virus spread to children quickly - all responses 
returning 403 by 9:12am
9) 302 responses for directory requests not ending with / were somehow 
able to be accomplished after the 403 virus had spread
10) none of the 403 responses had the IP number translated to a hostname
11) error log has lots of errors around the time 403's started with 
    "httpd: could not create IPC pipe".  After the first one appeared, then
    I started getting the "403" errors recorded to the error log.

For those with hyperreal accounts, the damage can be found in

/usr/local/www.tools/apache/logs/error_log
/var/log/httpd (today)
/export/logarchives/1995/06/19.httpd.gz (tomorrow)

If anyone can suggest anything else that needs to be looked for when this 
happens, let me know.  Hopefully this report has enough details to let 
those who know the IPC code find the problem.

	Brian


--=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=--
brian@organic.com  brian@hyperreal.com  http://www.[hyperreal,organic].com/