httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brian Akins <>
Subject Re: [PATCH] Re: Pluggable mod_log_config
Date Tue, 04 Oct 2005 13:15:32 GMT
Colm MacCarthaigh wrote:
> On Tue, Oct 04, 2005 at 08:37:08AM -0400, Brian Akins wrote:
> No, I don't think it's so simple at all. Although it would have to parse
> the SQL at init time (how does LogFormat know to ask the MySQL provider
> to do this?) 

LogFormat doesn't, but the init of the provider could.

that can't be simply untied from the actions at log-time;
> in the case of SQL for example you have to be very pedantic about the
> escaping, so that a request for "/foo\'; nasty sql;" doesn't kill us. 

That what the binding ensures.

> And we still need additional per-provider directives to provide the
> database, hostname, username and so on information. 

No, the init of the provider worries about the parameters.  In fact, the 
mysql modules may just have seperate directives to set this:

SqlLoggerUsername User
SqlLoggerPassword password

If such directives
> are required anyway, what effort are we saving? Why not just replace
> mod_log_config rather than plug into it. 

Because mod_log_config handles a bunch of stuff for us.  May be a better 
solution would be to use the standard log_config way of replacing init 
and writer and replace them with a pluggable one.

> Though if all of this text parsing is getting expensive, I wonder would
> anyone be interested in a protocol for binary logging from httpd?

Hmmm. interesting. Especially for things like spread, this would make 
alot of sense.

>>CustomLog /logs/site.sock common
> You re-implemented syslog :-) I've done the same myself for our hosting
> service, we use syslog-ng to do all sorts of weird things with the info. 

but syslog can't, on it's own, determine between virtuals (same problems 
as piped loggers).  I may have to look at syslog-ng.  Can you maybe 
off-list share some of your techniques?

>>define "damn busy." 
> We frequently see over 4000 requests per second being logged. Almost any
> time there's a major security update for fedora really. 

Okay, you qualify :)

> Pipes themselves have little overhead, it's basically shared memory with
> a standard IO interface and automatic mutexing. The processes on the
> other side certainly need not be cumbersome - and I really like that you
> can run a pipe-logger as a different uid, in a chroot and so on, it's a
> nice place to sandbox all of that icky SQL parsing and that kind of
> thing.

True.  The test parsing just seems so "icky" to determine the virtual 
host since Apache already knows that.

Brian Akins
Lead Systems Engineer
CNN Internet Technologies

View raw message