httpd-cvs mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From s.@apache.org
Subject svn commit: r1209743 - in /httpd/httpd/trunk: ./ docs/log-message-tags/ include/
Date Fri, 02 Dec 2011 22:26:55 GMT
Author: sf
Date: Fri Dec  2 22:26:54 2011
New Revision: 1209743

URL: http://svn.apache.org/viewvc?rev=1209743&view=rev
Log:
Add APLOGNO() macro for unique tags for every log message.
Add some scripts to make adding these tags easier.

Added:
    httpd/httpd/trunk/docs/log-message-tags/
    httpd/httpd/trunk/docs/log-message-tags/README   (with props)
    httpd/httpd/trunk/docs/log-message-tags/find-messages.cocci
    httpd/httpd/trunk/docs/log-message-tags/macros.h   (with props)
    httpd/httpd/trunk/docs/log-message-tags/next-number
    httpd/httpd/trunk/docs/log-message-tags/update-log-msg-tags
Modified:
    httpd/httpd/trunk/Makefile.in
    httpd/httpd/trunk/include/ap_mmn.h
    httpd/httpd/trunk/include/http_log.h

Modified: httpd/httpd/trunk/Makefile.in
URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/Makefile.in?rev=1209743&r1=1209742&r2=1209743&view=diff
==============================================================================
--- httpd/httpd/trunk/Makefile.in (original)
+++ httpd/httpd/trunk/Makefile.in Fri Dec  2 22:26:54 2011
@@ -242,5 +242,9 @@ suexec:
 x-local-distclean:
 	@rm -rf autom4te.cache
 
+update-log-tags update-log-msg-tags:
+	find server modules os -name \*.c|xargs perl \
+		docs/log-message-tags/update-log-msg-tags
+
 # XXX: This looks awfully platform-specific [read: bad form and style]
 include $(top_srcdir)/os/os2/core.mk

Added: httpd/httpd/trunk/docs/log-message-tags/README
URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/docs/log-message-tags/README?rev=1209743&view=auto
==============================================================================
--- httpd/httpd/trunk/docs/log-message-tags/README (added)
+++ httpd/httpd/trunk/docs/log-message-tags/README Fri Dec  2 22:26:54 2011
@@ -0,0 +1,53 @@
+This directory contains the infrastructure to create unique tags for error log
+messages. In the source, the tags use the APLOGNO(02182) macro where the argument
+is a 5 digit decimal number. The macro expands to "AH02182: ".
+
+The idea is that these tags help users finding useful information with search
+engines. The use of the macro is intended to prevent search engine hits at
+svn.apache.org or the svn commit mailing list.
+
+Basic rules:
+
+- Only messages of level debug and higher should get tags.
+- If the same message is logged at two different places in httpd, assign two
+  different tags.
+- If the context changes, where a message is generated, assign a new tag.
+- If only the text of the message changes (e.g. making it more verbose), keep
+  the tag.
+- Never reuse tags that have been removed, numbers are cheap.
+- Use the same tags in different branches.
+- The tag AH02182 is reserved for examples.
+- Currently only modules included in the httpd distribution should do this.
+
+TODO: Define what third-party modules should do.
+
+How to add a few new tags:
+==========================
+
+When adding new error messages, it is easiest to just add empty APLOGNO() tags
+and then run the update-log-msg-tags perl script on the source file. This will
+look into docs/log-message-tags/next-number to determine the next tag to be
+used, fill in all empty APLOGNO() tags, and update
+docs/log-message-tags/next-number accordingly.
+
+The toplevel Makefile has a target update-log-msg-tags to run run the script
+over all *.c files in the source tree.
+
+The script also puts a list of all messages in docs/log-message-tags/list.
+This list should not be committed to svn.
+
+
+How to add lots of new tags:
+============================
+
+In order to find candidate calls to ap_log_*error, coccinelle's spatch command
+can be used.  It will add empty APLOGNO() tags where the loglevel is a
+constant and is of level debug or above. Then, update-log-msg-tags can be used
+to add numbers to the APLOGNO() tags. The invocation for spatch is:
+
+DIR=docs/log-message-tags
+spatch -sp_file $DIR/find-messages.cocci -in_place -macro_file $DIR/macros.h <file1>
...
+
+After the initial addition of tags, calling spatch on the same file again may
+be a bad idea. But it may still be useful when adding larger pieces of code to
+httpd.

Propchange: httpd/httpd/trunk/docs/log-message-tags/README
------------------------------------------------------------------------------
    svn:eol-style = native

Added: httpd/httpd/trunk/docs/log-message-tags/find-messages.cocci
URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/docs/log-message-tags/find-messages.cocci?rev=1209743&view=auto
==============================================================================
--- httpd/httpd/trunk/docs/log-message-tags/find-messages.cocci (added)
+++ httpd/httpd/trunk/docs/log-message-tags/find-messages.cocci Fri Dec  2 22:26:54 2011
@@ -0,0 +1,18 @@
+@r@
+expression rv, s;
+constant char [] format;
+identifier level ~= "^APLOG_\(EMERG\|ALERT\|CRIT\|ERR\|WARNING\|NOTICE\|INFO\|STARTUP\|DEBUG\)$";
+identifier fn ~= "^ap_log_\(\|r\|c\|p\)error$";
+
+@@
+        fn( APLOG_MARK ,
+(
+        level
+|
+        level|APLOG_NOERROR
+|
+        level|APLOG_STARTUP
+)
+        ,rv, s
++       , APLOGNO()
+        ,format, ...)

Added: httpd/httpd/trunk/docs/log-message-tags/macros.h
URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/docs/log-message-tags/macros.h?rev=1209743&view=auto
==============================================================================
--- httpd/httpd/trunk/docs/log-message-tags/macros.h (added)
+++ httpd/httpd/trunk/docs/log-message-tags/macros.h Fri Dec  2 22:26:54 2011
@@ -0,0 +1,23 @@
+#define AP_DECLARE(x)             x
+#define AP_DECLARE_NONSTD(x)      x
+#define AP_CORE_DECLARE(x)        x
+#define AP_CORE_DECLARE_NONSTD(x) x
+#define AP_LUA_DECLARE(x)         x
+#define APR_DECLARE(x)            x
+#define APR_DECLARE_NONSTD(x)     x
+#define APU_DECLARE(x)            x
+#define APU_DECLARE_NONSTD(x)     x
+#define PROXY_DECLARE(x)          x
+#define DAV_DECLARE(x)            x
+#define APREQ_DECLARE(x)          x
+#define APREQ_DECLARE_PARSER(x)   x
+
+#define AP_DECLARE_DATA
+#define AP_MODULE_DECLARE_DATA
+#define APR_DECLARE_DATA
+#define APR_MODULE_DECLARE_DATA
+#define APU_DECLARE_DATA
+#define DAV_DECLARE_DATA
+#define PROXY_DECLARE_DATA
+
+#define AP_DECLARE_MODULE(foo)  module foo##_module

Propchange: httpd/httpd/trunk/docs/log-message-tags/macros.h
------------------------------------------------------------------------------
    svn:eol-style = native

Added: httpd/httpd/trunk/docs/log-message-tags/next-number
URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/docs/log-message-tags/next-number?rev=1209743&view=auto
==============================================================================
--- httpd/httpd/trunk/docs/log-message-tags/next-number (added)
+++ httpd/httpd/trunk/docs/log-message-tags/next-number Fri Dec  2 22:26:54 2011
@@ -0,0 +1 @@
+1

Added: httpd/httpd/trunk/docs/log-message-tags/update-log-msg-tags
URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/docs/log-message-tags/update-log-msg-tags?rev=1209743&view=auto
==============================================================================
--- httpd/httpd/trunk/docs/log-message-tags/update-log-msg-tags (added)
+++ httpd/httpd/trunk/docs/log-message-tags/update-log-msg-tags Fri Dec  2 22:26:54 2011
@@ -0,0 +1,155 @@
+#!/usr/bin/perl
+
+use warnings;
+use strict;
+
+my $basedir = "docs/log-message-tags";
+my $serial_file = "$basedir/next-number";
+my $serial = read_serial($serial_file);
+my $orig_serial = $serial;
+my %tags;
+
+foreach my $file (@ARGV) {
+    if ($file !~ /\.c$/) {
+        print STDERR "Skipping non-C file $file\n";
+        next;
+    }
+    process($file);
+}
+write_file($serial_file, "$serial\n") if $serial != $orig_serial;
+
+my $list = "";
+foreach my $tag (sort keys %tags) {
+    my $d = $tags{$tag};
+    $list .= "$tag: $d->{file}:$d->{line}: $d->{msg}\n";
+}
+write_file("$basedir/list", $list);
+
+exit 0;
+
+sub process
+{
+    my $file = shift;
+
+    open(my $fh, "<", $file) or die "open $file: $!";
+    #print STDERR "processing $file\n";
+    my $line = <$fh>;
+    my $modified;
+    my $result = "";
+    while (defined $line) {
+        if ($line =~ s{APLOGNO\(\),?}{gen_tag($file)}e) {
+            $modified = 1;
+        }
+        if ($line =~ /APLOGNO\(\s*(\d{5})\s*\)/ ) {
+            my $lineno = $.;
+            my $tag = $1;
+            while (1) {
+                if ($line =~ s/.*?
+                               APLOGNO\(\s*
+                               (\d+)
+                               \s*\)
+                               (
+                                (?: [\s\n]*
+                                    (?:"
+                                       (?:\\"|[^"])+    # a string constant
+                                       "
+                                      |
+                                       \w+              # things like APR_SIZE_T_FMT
+                                    )
+                                )*                      # zero or more string fragments.
We allow
+                                                        # zero because some logging constructs
may
+                                                        # use things like:
+                                                        #     logno=APLOGNO(...);
+                                                        #     ap_log_...(..., "%s...", logno,
...)
+                               )
+                               [\s\n]*
+                               [,);:\\]                 # the "," before the next argument,
+                                                        # or the closing brace of ap_log...(),
+                                                        # or the end of a statement (if used
+                                                        # outside of ap_log_...),
+                                                        # or ":" in things like:
+                                                        #     cond ? "msg1" : "msg2",
+                                                        # or "\" at the end of a macro line
+                              //xs) {
+                    my $match = $&;
+                    note_tag($file, $lineno, $1, $2);
+                    $result .= $match;
+                    last;
+                }
+                else {
+                    my $next = <$fh>;
+                    defined $next or die "can't find end of format string in $file:$lineno";
+                    $line .= $next;
+                    if ($next =~ /^#/) {
+                        # log format inside preprocessor #if, that's too complicated
+                        note_tag($file, $lineno, $tag, "");
+                        $result .= $line;
+                        $line = "";
+                        last;
+                    }
+                }
+            };
+        }
+        else {
+            $result .= $line;
+            $line = <$fh>;
+        }
+    }
+    close $fh;
+    write_file($file, $result) if $modified;
+}
+
+sub gen_tag
+{
+    my $file = shift;
+    my $msg = shift;
+    my $tag = sprintf('%05d', $serial++);
+    return "APLOGNO($tag)";
+}
+
+sub note_tag
+{
+    my $file = shift;
+    my $lineno = shift;
+    my $tag = shift;
+    my $msg = shift;
+
+    my $oneline = "";
+    while (length $msg) {
+        $msg =~ s/^[\s\n]+//s;
+        if ($msg =~ s{^"((?:\\"|[^"])+)"}{}) {
+            $oneline .= $1;
+        }
+        if ($msg =~ s{^(\w+)}{}) {
+            $oneline .= $1;
+        }
+    }
+    if (exists $tags{$tag}) {
+        print STDERR "WARNING: Duplicate tag $tag at $tags{$tag}->{file}:$tags{$tag}->{line}
and $file:$lineno\n";
+    }
+    $tags{$tag} = { file => $file, line => $lineno, msg => $oneline };
+}
+
+sub write_file
+{
+    my $file = shift;
+    my $data = shift;
+
+    my $tmpname = "$file.$$.tmp";
+
+    open(my $fh, ">", $tmpname) or die "open $tmpname: $!";
+    print $fh $data or die "write $tmpname: $!";
+    close($fh) or die "close $tmpname: $!";
+    rename($tmpname, $file) or die "rename $tmpname -> $file: $!";
+    print STDERR "Updated $file\n";
+}
+
+sub read_serial
+{
+    my $name = shift;
+    open(my $fh, "<", $name) or die "can't open $name, need to be started in the top source
dir";
+    my $num = <$fh>;
+    chomp $num;
+    $num =~ /^\d+$/ or die "invalid serial in $name: $num";
+    return $num;
+}

Modified: httpd/httpd/trunk/include/ap_mmn.h
URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/include/ap_mmn.h?rev=1209743&r1=1209742&r2=1209743&view=diff
==============================================================================
--- httpd/httpd/trunk/include/ap_mmn.h (original)
+++ httpd/httpd/trunk/include/ap_mmn.h Fri Dec  2 22:26:54 2011
@@ -372,6 +372,7 @@
  *                         add pool to ap_errorlog_info.
  * 20111201.0 (2.5.0-dev)  Add invalidate_entity() to the cache provider.
  * 20111202.0 (2.5.0-dev)  Use apr_status_t across mod_session API.
+ * 20111202.1 (2.5.0-dev)  add APLOGNO()
  */
 
 #define MODULE_MAGIC_COOKIE 0x41503234UL /* "AP24" */

Modified: httpd/httpd/trunk/include/http_log.h
URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/include/http_log.h?rev=1209743&r1=1209742&r2=1209743&view=diff
==============================================================================
--- httpd/httpd/trunk/include/http_log.h (original)
+++ httpd/httpd/trunk/include/http_log.h Fri Dec  2 22:26:54 2011
@@ -109,6 +109,14 @@ extern "C" {
 #endif
 
 /**
+ * APLOGNO() should be used at the start of the format string passed
+ * to ap_log_error() and friends. The argument must be a 5 digit decimal
+ * number. It creates a tag of the form "AH02182: "
+ * See docs/log-message-tags/README for details.
+ */
+#define APLOGNO(n)              "AH" #n ": "
+
+/**
  * APLOG_NO_MODULE may be passed as module_index to ap_log_error() and related
  * functions if the module causing the log message is not known. Normally this
  * should not be used directly. Use ::APLOG_MARK or ::APLOG_MODULE_INDEX



Mime
View raw message