perl-modperl mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Torsten Foertsch <>
Subject Re: sending all of stderr to a process
Date Mon, 06 Apr 2009 08:08:55 GMT
On Sun 05 Apr 2009, E R wrote:
> My concern is about what happens in this scenario:
> 1. Set everything up (named pipe, logger process, etc.)
> 2. Now suppose the logger process dies (it crashes or I need to
> restart it) 3. Any writes through existing connections to the named
> pipe will just fail, and I'll lose those writes.
> I'm pretty sure about #3, but I might be wrong.

A (named) pipe is a ring buffer within the kernel. A writer can write to 
it up to a certain level. Then it is "full" and the writer blocks or 
gets an EAGAIN for nonblocking writes.

So there is a certain period of time when the data resides in the kernel 
buffer. All that data is lost when the kernel panics, loses power or 

Then the reader reads a piece of data from the pipe and does something 
with it.

If that process crashes between reading the data from the pipe and 
finishing its processing that piece is lost.

So there are at least 2 periods of time when data can get lost. But is 
that all? It depends on your definition of "sending to stderr". For 
example lets look at httpd. The event it wants to log happens at a 
certain "point" (in quotes because it is a short period in reality) in 
time. Before it goes out of stderr there is a period of time when a 
string is made from the event in ap_log_error(), log_error_core() or 
somewhere else. That perhaps requires some malloc()ing to extend a pool 
etc. And of course there can be a stray SIGKILL on its way to the 
process that arrives just in the moment before the formatted string 
hits stderr.

That means there is a period of time when the event that is to be logged 
has happend but has not arrived on stderr.

All I want to say here is you can't make 100% sure that every event that 
has to be logged is really logged. You can achieve quite a good level 
but never 100%.

The good thing about a named pipe is that it can be connected by 
unrelated processes. So if the reader dies only the piece of data it is 
currently processing is lost. The writer can still write until the pipe 
is full. So, have your reader restarted fast enough. That's why I 
mentioned /etc/inittab.

> Perhaps I can't do what I want to do at the application level and I
> need to go to the kernel level - like write my own device driver or
> something, especially since I want to capture all output sent through
> stderr (ie. from Apache and from mod_perl and from any XS modules.)

As I said before you can't achieve 100% even at kernel level. And 
imagine you'd want to write to a data base from kernel space. I don't 
say its impossible but you'd have to reimplement the whole protocol and 
the protocol specs may be not even published. Happy hacking!

On Linux the ErrorLog of the base server is always opened on file 
descriptor 2 that means stderr. So, a reader that reads this stream 
gets everything (under normal conditions) that Apache, Perl or any XS 
module writes to stderr.


Need professional mod_perl support?
Just hire me:

View raw message