Date: Tue, 4 May 1999 00:22:36 -0700 (PDT)
From: Dean Gaudet <dgaudet@arctic.org>
To: new-httpd@apache.org
Subject: Re: fix for hybrid server problems.
In-Reply-To: <19990504020152.B27914@io.com>
Message-ID: <Pine.LNX.3.96dg4.990504000607.12702O-100000@twinlark.arctic.org>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Sender: new-httpd-owner@apache.org
Precedence: bulk
Reply-To: new-httpd@apache.org


On Tue, 4 May 1999, Manoj Kasichainula wrote:

> Can we rely on close() or shutdown() getting rid of our listening
> sockets while we're selecting on them?

The event thread is where we learn that we're supposed to do a graceful
shutdown... so it's trivial to make sure we don't select on them. 
Unfortunately we may need to either spawn a new thread to do call the
graceful shutdown methods, or we'll have to require the graceful shutdown
methods to be non-blocking.  Hmm. 

> > graceful restart and graceful shutdown are the two suggested forms
> > of restarting and shutting down the server.  They're the safe forms.
> 
> If we solve the above problem (and log rotation), this sounds
> reasonable.  Do we mind that there will be no way to get a mod_status
> display of our gracefully dying children after a graceful shutdown?

Didn't think about that... hmm. 

> 
> > logging:  yeah, there is difficulty with log rotation if graceful
> > restart is the only restart available -- there's no way for an
> > external program to know that all children have finished writing
> > the rotated logs.  There's a few possibilities for solving this...
> 
> Do the words "reliable", "piped", and "logs" get used, in that order?

That's one way yeah.  I think there are others.  Nothing makes me entirely
happy though -- hoping someone will come up with a killer solution.  The
one thing which irks me about piped logs is that many systems have small
pipe buffers.  What we really want is a deep buffer with priority
inversion -- force the log process to a lower priority, so it's only
awakened when the buffer is full.  Otherwise you incur extra context
switches.

I think you can tune the buffer size for socketpair()s on many systems... 
that's part of the solution. 

Maybe there's a slick solution using shared memory. 

If you want to log directly to files, you can use a lock to figure out if
any processes are currently writing to a log.  Open the log and put a
shared lock on it for each httpd child (locks are advisory on most
systems, use a separate lock file for systems with mandatory locks).  Then
the log rotator rotates the log, sends the graceful restart signal, and
tries to acquire an exclusive lock.  It won't acquire the lock until the
last child exits.

piped logs is probably a good method to start with. 

> > You'll be surprised to hear me bring up an acceptor thread again... but
> > I have a reason for returning its existance:  to service static requests.
> 
> It sounds cooler and cooler the more I think about it (which isn't
> much yet), but I really was hoping to get rid of that intervening
> request queue. :( Well, at least with only one thread doing the
> pushing on that queue, we can get rid of a pair of mutex calls.

I'm pretty sure we won't have much contention on that mutex... assume we
have one process per processor.  The mutex is local to a processor, and
the operations are short... 

The one which bothers me a bit more is the response queue which is a
pipe() -- that's two extra syscalls.  But they only happen on responses
longer than SO_SNDBUF bytes.  And I think it'll be worth it -- we can
handle far more long haul clients this way.

Dean