httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Darroch <>
Subject [PATCH] #39275 MaxClients on startup [Was: Bug in 2.0.56-dev]
Date Tue, 11 Apr 2006 20:50:57 GMT
Hi --

Alexander Lazic wrote:

>> After 'make install' i started apache, then some seconds later i got the
>> message '...MaxClients reached...' but there was no entry in the access
>> log, and nobody have make a request to this server.

Jeff Trawick wrote:

> There are problems accounting for child processes which are trying to
> initialize that result in the parent thinking it needs to create more
> children.  The less harmful flavor is when it thinks (incorrectly) it
> is already at MaxClients and issues the "reached MaxClients" message. 
> More disturbing is when MaxClients is very high and the parent keeps
> creating new children using exponential ramp-up.  That can be very
> painful.

   I have been seeing something similar with 2.2.0 using the worker
MPM, where with the following settings, I get over 10 child processes
initializing immediately (e.g., up to 15), and then they drop back to
10.  I see the "server reached MaxClients" message as well right
after httpd startup, although nothing is connecting yet.

<IfModule mpm_worker_module>
    StartServers         10
    MaxClients          150
    MinSpareThreads      25
    MaxSpareThreads     100
    ThreadsPerChild      10

   In my case, the problem relates to how long the child_init phase
takes to execute.  I can "tune" this by raising DBDMin (and DBDKeep)
so that mod_dbd attempts to open increasingly large numbers of
DB connections during child_init.  With DBDMin set to 0 or 1,
all is well; no funny behaviour.  Up at DBDMin and DBDKeep at 3,
that's when (for me) things go pear-shaped.

   In server/mpm/worker/worker.c, after make_child() creates a
child process it immediately sets the scoreboard parent slot's pid
value.  The main process goes into server_main_loop() and begins
executing perform_idle_server_maintenance() every second; this
looks at any process with a non-zero pid in the scoreboard and
assumes that any of its worker threads marked SERVER_DEAD are,
in fact, dead.

   However, if the child processes are starting "slowly" because
ap_run_child_init() in child_main() is taking its time, then
start_threads() hasn't even been run yet, so the threads aren't
marked SERVER_STARTING -- they're just set to 0 as the default
value.  But 0 == SERVER_DEAD, so the main process sees a lot
of dead worker threads and begins spawning new child processes,
up to MaxClients/ThreadsPerChild in the worst case.  In this case,
when no worker threads have started yet, but all possible child
processes have been spawned (and are working through their
child_init phases), then the following is true and the
"server reached MaxClients" message is printed, even though
the server hasn't started accepting connections yet:

    else if (idle_thread_count < min_spare_threads) {
        /* terminate the free list */
        if (free_length == 0) {

   I considered wedging another thread status into the
scoreboard, between SERVER_DEAD (the initial value) and
SERVER_STARTING.  The make_child() would set all the thread
slots to this value and start_threads() would later flip them
to SERVER_STARTING after actually creating the worker threads.

   That would have various ripple effects on other bits of
httpd, though, like mod_status and other MPMs, etc.  So instead
I tried adding a status field to the process_score scoreboard
structure, and making the following changes to worker.c such that
this field is set by make_child to SERVER_STARTING and then
changed to SERVER_READY once the start thread that runs
start_threads() has done its initial work.

   During this period, while the new child process is running
ap_run_child_init() and friends, perform_idle_server_maintenance()
just counts that child process's worker threads as all being
effectively in SERVER_STARTING mode.  Once the process_score.status
field changes to SERVER_READY, perform_idle_server_maintenance()
begins to look at the individual thread status values.

   Any thoughts?  The patch in Bugzilla doesn't address other
MPMs that might see the same behaviour (event, and maybe prefork?)

It also doesn't necessarily play ideally well with the fact that
new child processes can gradually take over thread slots in
the scoreboard from a gracefully exiting old process -- the
count of idle threads for that process will be pegged (only
by perform_idle_server_maintenance()) at ap_threads_per_child
until the new process creates its first new worker thread.
But, that may be just fine....  I'll keep poking around and
testing and maybe a better idea will present itself.


View raw message