httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefan Fritsch ...@sfritsch.de>
Subject Re: Enhanced error log format for trunk?
Date Tue, 08 Jun 2010 21:40:38 GMT
On Tuesday 08 June 2010, Rainer Jung wrote:
> On 08.06.2010 22:56, Stefan Fritsch wrote:
> > On Monday 07 June 2010, Rainer Jung wrote:
> >> On 03.06.2010 16:49, Stefan Fritsch wrote:
> >>> On Tuesday 01 June 2010, Rainer Jung wrote:
> >>>> 4) General correlation improvements
> >>>> 
> >>>> To be able to correlate error and access log, it would be
> >>>> helpful to share a common id, e.g. the unique_id, and be able
> >>>> to log it in both files. The id generated by mod_unique_id
> >>>> comes too late though (post_read_request). Since it actually
> >>>> only uses the request timestamp and the connection id of the
> >>>> request, it could be calculated much earlier.
> >>> 
> >>> I have thought about this before, but I wanted to get the
> >>> per-module loglevel configuration into trunk first.
> >>> 
> >>> The log id could be created by the first call to ap_log_rerror.
> >>> If there has not been an errorlog entry for a request by the
> >>> time of the log transaction hook, the corresponding field in
> >>> the access log would just be "-".
> >> 
> >> That's an interesting idea, making it pretty trivial to filter
> >> those lines, that have any error log entries.
> >> 
> >>> The function creating the log id could also log some
> >>> interesting information once per request, instead of logging
> >>> it for every log line. For example PID/TID, local+remote
> >>> IP+Port, number of keep-alives on the connection, ...
> >> 
> >> Hmmm, but some basics should stay on each line, otherwise it's
> >> to much hassle collection stuff (or we provide a script). Being
> >> able to correlate between access and error log is nice, being
> >> forced to correlate inside the error log to collect the data is
> >> not so nice.
> > 
> > The script would be "grep log-id error_log". I fear the useful
> > information depends very much on the situation. Logging
> > everything makes sure that we have the useful information. But
> > this would obviously flood the error log if we do it on every
> > line or even for every request. Hence my suggestion to only log
> > it for requests which have at least one other error log message
> > logged.
> > 
> > This should be configurable. Either this conditional verbose
> > logging or the "some information in every line" style that we
> > have now. And it should also be configurable what the "some
> > information" should be.
> > 
> >> Do you already have any code fragments for that?
> > 
> > No, these are only ideas so far. I have thought of following
> > things which may be nice to have as error log flags:
> > 
> > - timestamp granularity
> > - don't log to the error log file (useful for modules using the
> > error log hook)
> > - log some kind of unique id
> > - do conditional verbose logging as outlined above
> > - log the name of the module
> > - always or never log source file name+line number, or make it
> > depend on the log level like now
> > - log process/thread id
> > - log errors that are induced by the client at level error or at
> > level debug (e.g. file not found, connection closed by client)
> > 
> > Information that may be useful in the conditional verbose
> > logging: - local/remote port+ip,
> > - vhost/Host-header
> > - some way to correlate which requests are on the same
> > connection, like a connection id (I think pid/tid are not enough
> > with mpm event, but I could be mistaken)
> 
> Correct
> 
> > - Referrer-header
> > - number of keep-alive requests on the connection
> > 
> > That's a lot, but I can think of situations where any of the
> > above information would be useful. Actually, for most pieces I
> > can remember situations where the information would have been
> > useful.
> 
> What about introducing log formats for the error log, like
> mod_log_config does for the access log? Parsing the format string
> once and registering callbacks to do the actual formatting.

That would be the most flexible solution. To make me happy, there 
would be one format string for the actual line format, one that is 
logged once per connection, and one that is logged once per request.

> Naive implementation would mean a lot of code copying from
> mod_log_config.

There would be some added complications:
- log format items may need any of server_rec, conn_req, or 
request_req but it is possible that some or all of these are not 
available
- We cannot simply allocate memory from the server/connection/request 
pool. Either we allocate a subpool from the respective pool and 
destroy that after each line (but we don't even have a server_rec 
during startup), or we would need a dedicated logging pool (per 
thread?) that is cleared after each log line.

But it certainly sounds do-able.

Mime
View raw message