httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dean Gaudet <>
Subject reliable piped logs
Date Mon, 07 Jul 1997 02:11:09 GMT
So the reason I've hated to recommend using | to a logger before is
that it's totally unreliable.  If for some reason the child should die,
bad things happen.  Ideally the parent should be able to respawn the
logging child and no hits should be lost.

The children cannot do the spawning (there's the problem of
synchronization, and permissions problems), and they need some way to get
a new file handle for the new child.  Here's a few ways with the
pros and cons I can think of:

- use descriptor passing to pass handle to each child

    Given that NCSA uses descriptor passing, this works well on most
    architectures.  It's slow, but it's also not something that's done
    frequently.  It would require the parent to have a fd open to
    each child, damn near impossible if you ask me

- use a named pipe

    Child can reopen the named pipe if the logger dies.

    I'm not sure that you can get the "multiple-writers, one-reader"
    semantic on named pipes.  It seems to work in some cases (i.e. all
    writers blocked trying to open it before the reader opens it)...
    but fails in other cases.  So we'd probably have to hack it to have
    a named pipe for each child slot, and the logging child would need
    to open up each named pipe... and would have to always try to re-open
    them ... and wouldn't be able to do that without blocking, and would
    totally suck.

- use unix domain sockets

    Child can reopen the socket if the logger dies.

    Similar problems to the last one when using SOCK_STREAM sockets --
    the logger needs to do listen()/accept() crap on the unix socket
    and have a fd for each child.  But SOCK_DGRAM offers some hope.
    The logger would just read() datagrams, not caring who sent them.

    Unix domain sockets (and probably named pipes) have some interesting
    problems on various unixes -- regardless of the file owner/group and
    permission bits any process can open the socket/pipe.  The only way
    to protect them is to nest them in a directory with the appropriate
    permissions.  This is a very common problem (present in BSD up through
    most of the 4.3s I think).  Regardless, the uid the children runs
    as need to be able to write the socket ...  which is bad because
    then CGIs could potentially write the socket.

    SOCK_DGRAM isn't reliable, but in practice only loses packets (in
    unix domain) if the socket runs out of buffer space ... i.e. the log
    child takes too long to read.

- un-named pipes and graceful restart

    Proceed as we do now and if the logger dies then perform a
    graceful restart.  Will lose the log of all the hits that children
    are attempting to log when the graceful hits.

    This could be helped by having an "overflow" file, much like how INN
    handles its feeds.  When a child discovers that the logger is dead,
    or that it would have to block while writing to the logger, then write
    the data to another file handle into an overflow file.  The parent
    opens both the logger pipe and the overflow file at init time.

    There's a bit of work to properly get the logger to handle the overflow
    file ... it's not easy (or desirable imho) to get db-like reliability
    here, but it's possible to get a bit better than nothing.

I'm partial to the last one.

Thoughts?  Additions?


View raw message