httpd-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Hammerman <jhammer...@videoegg.com>
Subject RE: [users@httpd] Re: mod_log_config issue
Date Fri, 29 Jan 2010 19:43:06 GMT
Hey all. First of all thanks for the assistance.

I haven't resolved the issue, however I thought I would add some more detail regarding my
investigations, if anyone ever digs through these archives.

It seems that sed doesn't actually flush the stream it is editing until it hits a '\n'. So
I bet sed is editing the stream in place, and storing all of its edits in memory.

LogFormat supports the '\n' character - but I still can't get sed to actually write out to
disk.

-----Original Message-----
From: Joe Hammerman [mailto:jhammerman@videoegg.com] 
Sent: Thursday, January 28, 2010 11:47 AM
To: users@httpd.apache.org
Subject: RE: [users@httpd] Re: mod_log_config issue

Hi Dan,

When the sed command is replaced with /bin/cat, logs are generated. Using a sed command that
does nothing results in no log output; e.g.

CustomLog "| /bin/sed s/// |/usr/bin/cronolog..."

Or

CustomLog "| /bin/sed 's///' | /usr/bin/cronolog..."

Or

CustomLog "| /bin/sed -e 's///' | /usr/bin/cronolog..."

:/

I have tried using a wrapper script. Here it is in its original form:

#!/bin/bash

cat - | while read LINE
do
echo ${LINE} |
/bin/sed s/[0-9]\\\{1,3\\\}\\\.[0-9]\\\{1,3\\\}\\\.[0-9]\\\{1,3\\\}\\\.[0-9]\\\{1,3\\\},\\\
//g | /usr/bin/sudo -u VEsvc /usr/sbin/cronolog --period=1hours /mnt/export/www/logs/beacon/%Y%m%d/%H/access_log
done

and then, per your suggestion:

#!/bin/bash

#cat - | while read LINE
#do
#echo ${LINE} |
/bin/sed s/[0-9]\\{1,3\\}\\.[0-9]\\{1,3\\}\\.[0-9]\\{1,3\\}\\.[0-9]\\{1,3\\},\\
//g | /usr/bin/sudo -u VEsvc /usr/sbin/cronolog --period=1hours /mnt/export/www/
logs/beacon/%Y%m%d/%H/access_log
#done


httpd.conf was configured as you said.

In either case, no logs are generated.

-----Original Message-----
From: news [mailto:news@ger.gmane.org] On Behalf Of Dan Poirier
Sent: Thursday, January 28, 2010 7:10 AM
To: users@httpd.apache.org
Subject: [users@httpd] Re: mod_log_config issue

Joe Hammerman <jhammerman@videoegg.com> writes:

> 	If we replaced Sed with Cat, I'm a little confused as to what we would be catting; there's
a stream coming in, right?
>
> To your second question - yes, logging is fully functional with sudo
>
> To your final question - no, even with a sed command that performs no actions, no logging
information is generated.
>
> -----Original Message-----
> From: news [mailto:news@ger.gmane.org] On Behalf Of Dan Poirier
> Sent: Wednesday, January 27, 2010 12:03 PM
> To: users@httpd.apache.org
> Subject: [users@httpd] Re: mod_log_config issue
>
>> We have an issue with mod_log_config; specifically we are trying to pipe log output
through Sed before it goes to Cronolog. The result is that we get no output whatsoever.
>>
>>     CustomLog "| /bin/sed s/[0-9]\\\{1,3\\\}\\\.[0-9]\\\{1,3\\\}\\\.[0-9]\\\{1,3\\\}\\\.[0-9]\\\{1,3\\\},\\\
//g | /usr/bin/sudo -u VEsvc /usr/sbin/cronolog --period=1hours /mnt/export/www/logs/beacon/%Y%m%d/%H/survey_log"
combined env=survey_log
>>
>>     ErrorLog "| /bin/sed s/[0-9]\\\{1,3\\\}\\\.[0-9]\\\{1,3\\\}\\\.[0-9]\\\{1,3\\\}\\\.[0-9]\\\{1,3\\\},\\\
//g | /usr/bin/sudo -u VEsvc /usr/sbin/cronolog --period=1hours /mnt/export/www/logs/beacon/%Y%m%d/%H/error_log"
>>
>> We have also experimented with writing a wrapper script that performs all three of
the above functions - the result is the same.
>>
> What's the simplest case that doesn't work?  What if you take out the
> whole 'sed' command and just use /bin/cat?  Does invoking cronolog with
> sudo work when not receiving input piped from another command?  Does it
> work with a simpler sed script?

Okay, then you're saying this gives you logging:

     CustomLog "| /usr/bin/sudo -u VEsvc /usr/sbin/cronolog --period=1hours /mnt/export/www/logs/beacon/%Y%m%d/%H/survey_log"
combined env=survey_log

but this doesn't:

     CustomLog "| /bin/sed | /usr/bin/sudo -u VEsvc /usr/sbin/cronolog --period=1hours /mnt/export/www/logs/beacon/%Y%m%d/%H/survey_log"
combined env=survey_log

How about this:

     CustomLog "| /bin/cat | /usr/bin/sudo -u VEsvc /usr/sbin/cronolog --period=1hours /mnt/export/www/logs/beacon/%Y%m%d/%H/survey_log"
combined env=survey_log

"cat" with no arguments just copies stdin to stdout, so we can tell if sed is the problem,
or the piping.

Also, I would think if putting the piped command directly in the log config is the problem,
then replacing it with a wrapper script ought to work.  I assume your script looked like:

#!/bin/sh
/bin/sed s/[0-9]\\\{1,3\\\}\\\.[0-9]\\\{1,3\\\}\\\.[0-9]\\\{1,3\\\}\\\.[0-9]\\\{1,3\\\},\\\
//g | /usr/bin/sudo -u VEsvc /usr/sbin/cronolog --period=1hours /mnt/export/www/logs/beacon/%Y%m%d/%H/survey_log

(maybe with one level of backslashes removed), was marked executable, and the full path was
configured:

CustomLog "| /path/to/wrapper/script" combined env=survey_log

You might try a simpler script:

#!/bin/sh
/bin/cat >>/path/to/logfile

again just to rule out something funny in the sed/sudo/cronolog part as opposed to Apache.


---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
   "   from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
   "   from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
   "   from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


Mime
View raw message