httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From r..@engelschall.com (Ralf S. Engelschall)
Subject [PATCH] Network Socket Support for Logfiles
Date Thu, 05 Mar 1998 15:00:29 GMT

Network Socket Support for Logfiles
-----------------------------------

Background:
  In July 1997 Stanley Gambarin <stanleyg@cs.bu.edu> posted a short patch for
  the http_log.c file for hacking in the ability to use the syntax
  hostname:port for the filename argument of ErrorLog which leads to the open
  of a socket to this target. This is a very useful variant of writing out log
  information, especially for ISPs or large website providers who use a
  cluster of webservers. Here all Apache servers just log their information to
  a master machine, providing you the ability to monitor all websites at a
  single point.

Problem of the past:
  Although this variant of logging is very useful for a lot of people, we
  didn't incorporated it, because the patch was not very complete. The
  original patch from Stanley directly changes the open_error_log() function
  in src/main/http_log.c which leads to the fact that network sockets could
  only be used for the ErrorLog directive. And it didn't documented the new
  possible variant. 

Solution:
  I've started from scratch today and first wrote a open_socket_log()
  function in addition to the already existing open_piped_log() function.
  Then I've just _minimally_ patched mod_log_config.c and the open_error_log()
  function in http_log.c to use this new open_socket_log() function. Finally
  the corresponding documentation changes are provided to document this
  variant of writing logfiles.

Effect:
  For instance when you now have a tool like "socket" or "netcat" installed
  (FreeBSD and Linux hackers usually have out-of-the-box), you just use
     $ socket -f -l -s 8000 >>/path/to/master/access_log &
     $ socket -f -l -s 8001 >>/path/to/master/error_log &
  on the monitoring site `mastersite' and 
     TransferLog @mastersite:8000
     ErrorLog    @mastersite:8001
  on all Apache's on your cluster and then magically all log informations of
  the whole cluster is collected at the master site.

BTW: This is also very useful when using Apache in Reverse Proxy context.
     Both for writing the Reverse Proxy delegation logfile to the monitoring
     site via
         CustomLog @master:8000 "%{%v/%T}t %h -> %{SERVER}e URL: %U"
     and to collect the logfiles of the various back-end servers as
     described above.

                                       Ralf S. Engelschall
                                       rse@engelschall.com
                                       www.engelschall.com
Index: src/CHANGES
===================================================================
RCS file: /e/apache/REPOS/apache-1.3/src/CHANGES,v
retrieving revision 1.687
diff -u -r1.687 CHANGES
--- CHANGES	1998/03/04 14:28:25	1.687
+++ CHANGES	1998/03/05 14:36:10
@@ -1,5 +1,12 @@
 Changes with Apache 1.3b6
 
+  *) Add the ability to write logfile entries to network sockets in addition
+     to local files and pipes for ErrorLog (core) and TransferLog/CustomLog
+     (mod_log_config). This is intended to be used either for clusters of
+     webservers where one wants to store all error and transfer log
+     information on a single master site for monitoring or just to on-the-fly
+     transfer away log information. [Ralf S. Engelschall]
+
   *) Fix mod_rewrite for the ugly API case where <VirtualHost> sections exist
      but without any RewriteXXXXX directives. Here mod_rewrite is given no
      chance by the API to initialise its per-server configuration and thus
Index: src/include/http_log.h
===================================================================
RCS file: /e/apache/REPOS/apache-1.3/src/include/http_log.h,v
retrieving revision 1.25
diff -u -r1.25 http_log.h
--- http_log.h	1998/01/21 19:17:38	1.25
+++ http_log.h	1998/03/05 13:50:43
@@ -132,4 +132,6 @@
 #define piped_log_write_fd(pl)	(fileno((pl)->write_f))
 #endif
 
+API_EXPORT(int) open_socket_log(char *target);
+
 #endif	/* !APACHE_HTTP_LOG_H */
Index: src/main/http_log.c
===================================================================
RCS file: /e/apache/REPOS/apache-1.3/src/main/http_log.c,v
retrieving revision 1.48
diff -u -r1.48 http_log.c
--- http_log.c	1998/01/07 16:46:05	1.48
+++ http_log.c	1998/03/05 14:37:07
@@ -213,6 +213,11 @@
 	s->error_log = NULL;
     }
 #endif
+
+    else if (*s->error_fname == '@' && strchr(s->error_fname, ':') != NULL)
{
+        s->error_log = fdopen(open_socket_log(s->error_fname+1), "w");
+    }
+
     else {
 	fname = server_root_relative (p, s->error_fname);
         if(!(s->error_log = pfopen(p, fname, "a"))) {
@@ -638,3 +643,56 @@
     pfclose (pl->p, pl->write_f);
 }
 #endif
+
+
+API_EXPORT(int) open_socket_log(char *target)
+{
+    char host[MAX_STRING_LEN];
+    char *port;
+    struct hostent *he;
+    struct protoent *pe;
+    struct sockaddr_in sar;
+    FILE *fp;
+    int s;
+
+    /* parse host:port pair */
+    ap_cpystrn(host, target, sizeof(host));
+    if ((port = strchr(host, ':')) == NULL) {
+        fprintf(stderr, "httpd: Invalid socket target `%s'\n", target);
+        exit(1);
+    }
+    *port++ = '\0';
+
+    /* create socket address structure */
+    memset((char *)&sar, 0, sizeof(sar));
+    sar.sin_family = AF_INET;
+    sar.sin_port   = htons(atoi(port));
+    if ((sar.sin_addr.s_addr = inet_addr(host)) == INADDR_NONE) {
+        if ((he = gethostbyname(host)) == NULL) {
+            perror("gethostbyname");
+            fprintf(stderr, "httpd: Couldn't resolve hostname `%s'\n", host);
+            exit(1);
+        }
+        sar.sin_addr.s_addr = *((u_long *)(he->h_addr_list[0]));
+    }
+
+    /* create the socket and connect to it */
+    if ((pe = getprotobyname("tcp")) == NULL) {
+        perror("getprotobyname");
+        fprintf(stderr, "httpd: Couldn't resolve TCP protocol info\n");
+        exit(1);
+    }
+    if ((s = socket(AF_INET, SOCK_STREAM, pe->p_proto)) < 0) {
+        perror("socket");
+        fprintf(stderr, "httpd: Couldn't create socket\n");
+        exit(1);
+    } 
+    if (connect(s, (struct sockaddr *)&sar, sizeof(sar)) < 0) {
+        perror("connect");
+        fprintf(stderr, "httpd: Couldn't connect to %s\n", target);
+        exit(1);
+    }
+
+    return s;
+}
+
Index: src/modules/standard/mod_log_config.c
===================================================================
RCS file: /e/apache/REPOS/apache-1.3/src/modules/standard/mod_log_config.c,v
retrieving revision 1.47
diff -u -r1.47 mod_log_config.c
--- mod_log_config.c	1998/03/03 08:31:28	1.47
+++ mod_log_config.c	1998/03/05 14:21:35
@@ -912,6 +912,9 @@
         }
         cls->log_fd = piped_log_write_fd(pl);
     }
+    else if (*cls->fname == '@' && strchr(cls->fname, ':') != NULL) {
+        cls->log_fd = open_socket_log(cls->fname+1);
+    }
     else {
         char *fname = server_root_relative(p, cls->fname);
         if ((cls->log_fd = popenf(p, fname, xfer_flags, xfer_mode)) < 0) {
Index: htdocs/manual/mod/core.html
===================================================================
RCS file: /e/apache/REPOS/apache-1.3/htdocs/manual/mod/core.html,v
retrieving revision 1.103
diff -u -r1.103 core.html
--- core.html	1998/02/20 06:52:03	1.103
+++ core.html	1998/03/05 14:27:26
@@ -775,6 +775,8 @@
 then it is assumed to be relative to the <A HREF="#serverroot">ServerRoot</A>.
 If the filename begins with a pipe (|) then it is assumed to be a command to
 spawn to handle the error log.
+If the filename begins with an at-sign (@) then it is assumed to be followed
+by a <CODE>hostname:port</CODE> pair for opening a socket.
 <p>
 Example:
 <BLOCKQUOTE><CODE>ErrorLog /dev/null</CODE></BLOCKQUOTE>
Index: htdocs/manual/mod/mod_log_config.html
===================================================================
RCS file: /e/apache/REPOS/apache-1.3/htdocs/manual/mod/mod_log_config.html,v
retrieving revision 1.26
diff -u -r1.26 mod_log_config.html
--- mod_log_config.html	1998/03/03 08:38:26	1.26
+++ mod_log_config.html	1998/03/05 14:26:24
@@ -365,6 +365,8 @@
 <DD>A program to receive the agent log information on its standard input.
 Note the a new program will not be started for a VirtualHost if it inherits
 the TransferLog from the main server.
+<DT> `@' followed by a hostname:port pair
+<DD>A socket to receive the log information.
 </DL>
 <STRONG>Security:</STRONG> if a program is used, then it will be
 run under the user who started httpd. This will be root if the server

Mime
View raw message