www-apache-bugdb mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pekka Savola <pek...@netcore.fi>
Subject general/7570: under heavy traffic, root process dies with 'LOCK_EX: Error getting accept lock. Exiting!'
Date Mon, 16 Apr 2001 16:41:44 GMT

>Number:         7570
>Category:       general
>Synopsis:       under heavy traffic, root process dies with 'LOCK_EX: Error getting accept
lock. Exiting!'
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    apache
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   apache
>Arrival-Date:   Mon Apr 16 09:50:00 PDT 2001
>Closed-Date:
>Last-Modified:
>Originator:     pekkas@netcore.fi
>Release:        1.3.19
>Organization:
apache
>Environment:
FreeBSD xxx.org 4.2-STABLE FreeBSD 4.2-STABLE #0: Tue Jan 23 22:28:08 PST 2001     root@xxx.org:/usr/obj/usr/src/sys/XXX
 i386

Rather basic FreeBSD 4.2-STABLE environment, syncronized like 40 days ago.

All the data is on local HDD disk.
>Description:
The site is running 500-600 simultaneous users (httpd processes) on P3-500/512 MB.

Normally everything runs smooth.  However, once a week there is a scheduled
log resolving event; about 1 GB worth of access logs are being resolved.

When this begins, master httpd process reproducibly dies with:

[Mon Apr 16 08:30:11 2001] [emerg] (9)Bad file descriptor: flock: LOCK_EX: Error getting accept
lock. Exiting!
[Mon Apr 16 08:30:12 2001] [alert] Child 88208 returned a Fatal error... 

This should not be an issue with network buffers, or the like as there is ample capacity left:

# netstat -m
6320/12496/65536 mbufs in use (current/peak/max):
        5363 mbufs allocated to data
        957 mbufs allocated to packet headers
5068/10094/16384 mbuf clusters in use (current/peak/max)
23312 Kbytes allocated to network (47% of mb_map in use)
0 requests for memory denied
0 requests for memory delayed
0 calls to protocol drain routines

LockFile is not defined in httpd.conf; this should not be a problem here though.

The apache is compiled using FreeBSD ports collection defaults (except for HARD_SERVER_LIMIT=1024),
significantly:
--
 -D HAVE_MMAP
 -D USE_MMAP_SCOREBOARD
 -D USE_MMAP_FILES
 -D USE_FLOCK_SERIALIZED_ACCEPT
 -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
--

For the log-resolving process, jdresolve (http://freshmeat.net/projects/jdresolve/).
>How-To-Repeat:
Might be difficult.

A lot of requests + very heavy non-apache -related traffic (e.g. DNS lookups 50/sec parallelized)
might do the trick.
>Fix:
I'm testing whether -DUSE_FCNTL_SERIALIZED_ACCEPT instead of FLOCK
would do the trick, but if it works, there might be a more widespread
problem afoot.
>Release-Note:
>Audit-Trail:
>Unformatted:
 [In order for any reply to be added to the PR database, you need]
 [to include <apbugs@Apache.Org> in the Cc line and make sure the]
 [subject line starts with the report component and number, with ]
 [or without any 'Re:' prefixes (such as "general/1098:" or      ]
 ["Re: general/1098:").  If the subject doesn't match this       ]
 [pattern, your message will be misfiled and ignored.  The       ]
 ["apbugs" address is not added to the Cc line of messages from  ]
 [the database automatically because of the potential for mail   ]
 [loops.  If you do not include this Cc, your reply may be ig-   ]
 [nored unless you are responding to an explicit request from a  ]
 [developer.  Reply only with text; DO NOT SEND ATTACHMENTS!     ]
 
 


Mime
View raw message