httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christian von Roques <roq...@mti.ag>
Subject [PATCH] MaxBusyKeepAlive graceful degradation for masses of persisten connections
Date Sun, 06 Aug 2000 23:09:55 GMT
We're expecting to handle masses of long-lived persistent HTTPS/1.1
connections with some requests every few seconds in the near future.
The problem I see with this is that each persistent connection ties up
one server process and we'll likely hit the MaxClients limit which
will result in un-accept()ed connections and locked out clients.

The solution I've currently planned is a graceful degradation of
service from persistent HTTPS/1.1 connections to non-persistent
HTTPS/1.1 connections.  Each server process will check if we're near
MaxClients and not honor requests for persistence of the current
connection, but close it after the request has been handled.

I've implemented this [patch against Apache-1.3.13.dev from CVS
below], but there are some issues about this implementation I don't
really like:

 * The names MaxBusyKeepAlive and max_busy_keep_alive are ugly and
   not really self explanatory.  max_keep_alives is even worse.

 * ap_busy_servers() is implemented in http_main.c, because it uses
   max_daemons_limit, its only use is in http_protocol.c

 * The scoreboard will be traversed for each request, which is what
   Dean worked against in Apache-1.3a1.

Inspite of these issues, the patch works for my tests and stops my
test-server from locking out clients when bombarded with two times
MaxClients clients all trying to make long-lived persistent
connections.

I've thought about sending the server process in SERVER_BUSY_KEEPALIVE
state with the soonest to fire timeout a SIG_ALARM, but that would
probably allow for some `interesting' race-conditions with the process
running perform_idle_server_maintenance().

Do you have any suggestions for improvements, or is it just to ugly?

        Christian.


diff -wurdX nodiff.pats apache-1.3.dev/conf/httpd.conf-dist apache-1.3.dev+MaxBusyKeepAlive/conf/httpd.conf-dist
--- apache-1.3.dev/conf/httpd.conf-dist	Thu Jun  8 10:25:16 2000
+++ apache-1.3.dev+MaxBusyKeepAlive/conf/httpd.conf-dist	Sat Aug  5 18:30:31 2000
@@ -116,6 +116,17 @@
 MaxKeepAliveRequests 100
 
 #
+# MaxBusyKeepAlive: The maximum number of busy server processes upto
+# which persistence of connections is honored.  If you happen to need
+# more server processes, requests are still handled, but servers deny
+# to be tied up waiting for further requests on a connection and close
+# it immediately after each request to be free to serve other
+# connections.  A good value might be: MaxClients - 1.5*MaxSpareServers
+# The default is 0, which places no limit on persistent connections.
+#
+MaxBusyKeepAlive 0
+
+#
 # KeepAliveTimeout: Number of seconds to wait for the next request from the
 # same client on the same connection.
 #
diff -wurdX nodiff.pats apache-1.3.dev/src/include/http_main.h apache-1.3.dev+MaxBusyKeepAlive/src/include/http_main.h
--- apache-1.3.dev/src/include/http_main.h	Thu Jun 15 15:18:26 2000
+++ apache-1.3.dev+MaxBusyKeepAlive/src/include/http_main.h	Sun Aug  6 18:13:39 2000
@@ -123,6 +123,7 @@
 API_EXPORT(void) ap_child_terminate(request_rec *r);
 API_EXPORT(void) ap_sync_scoreboard_image(void);
 int ap_update_child_status(int child_num, int status, request_rec *r);
+int ap_busy_servers(void);
 void ap_time_process_request(int child_num, int status);
 unsigned int ap_set_callback_and_alarm(void (*fn) (int), int x);
 API_EXPORT(int) ap_check_alarm(void);
diff -wurdX nodiff.pats apache-1.3.dev/src/include/httpd.h apache-1.3.dev+MaxBusyKeepAlive/src/include/httpd.h
--- apache-1.3.dev/src/include/httpd.h	Mon Jul 17 10:13:04 2000
+++ apache-1.3.dev+MaxBusyKeepAlive/src/include/httpd.h	Sat Aug  5 18:17:58 2000
@@ -274,6 +274,13 @@
 #define DEFAULT_KEEPALIVE 100
 #endif
 
+/* Number of busy server processes upto which persistence of connections
+ * is honored, 0 for unlimited.
+ */
+#ifndef DEFAULT_MAX_BUSY_KEEPALIVE
+#define DEFAULT_MAX_BUSY_KEEPALIVE 0
+#endif
+
 /* The size of the server's internal read-write buffers */
 #define IOBUFSIZE 8192
 
@@ -942,6 +949,7 @@
     int timeout;		/* Timeout, in seconds, before we give up */
     int keep_alive_timeout;	/* Seconds we'll wait for another request */
     int keep_alive_max;		/* Maximum requests per connection */
+    int max_busy_keep_alive;	/* Maximum busy servers for keep-alives */
     int keep_alive;		/* Use persistent connections? */
     int send_buffer_size;	/* size of TCP send buffer (in bytes) */
 
diff -wurdX nodiff.pats apache-1.3.dev/src/main/http_config.c apache-1.3.dev+MaxBusyKeepAlive/src/main/http_config.c
--- apache-1.3.dev/src/main/http_config.c	Thu Jun  1 19:42:24 2000
+++ apache-1.3.dev+MaxBusyKeepAlive/src/main/http_config.c	Sat Aug  5 18:18:28 2000
@@ -1346,6 +1346,7 @@
     s->keep_alive_timeout = 0;
     s->keep_alive = -1;
     s->keep_alive_max = -1;
+    s->max_busy_keep_alive = -1;
     s->error_log = main_server->error_log;
     s->loglevel = main_server->loglevel;
     /* useful default, otherwise we get a port of 0 on redirects */
@@ -1405,6 +1406,9 @@
 	if (virt->keep_alive_max == -1)
 	    virt->keep_alive_max = main_server->keep_alive_max;
 
+	if (virt->max_busy_keep_alive == -1)
+	    virt->max_busy_keep_alive = main_server->max_busy_keep_alive;
+
 	if (virt->send_buffer_size == 0)
 	    virt->send_buffer_size = main_server->send_buffer_size;
 
@@ -1465,6 +1469,7 @@
     s->timeout = DEFAULT_TIMEOUT;
     s->keep_alive_timeout = DEFAULT_KEEPALIVE_TIMEOUT;
     s->keep_alive_max = DEFAULT_KEEPALIVE;
+    s->max_busy_keep_alive = DEFAULT_MAX_BUSY_KEEPALIVE;
     s->keep_alive = 1;
     s->next = NULL;
     s->addrs = ap_pcalloc(p, sizeof(server_addr_rec));
diff -wurdX nodiff.pats apache-1.3.dev/src/main/http_core.c apache-1.3.dev+MaxBusyKeepAlive/src/main/http_core.c
--- apache-1.3.dev/src/main/http_core.c	Mon Feb 28 09:42:24 2000
+++ apache-1.3.dev+MaxBusyKeepAlive/src/main/http_core.c	Sat Aug  5 18:17:58 2000
@@ -2083,6 +2083,17 @@
     return NULL;
 }
 
+static const char *set_max_busy_keep_alive(cmd_parms *cmd, void *dummy, char *arg) 
+{
+    const char *err = ap_check_cmd_context(cmd, NOT_IN_DIR_LOC_FILE|NOT_IN_LIMIT);
+    if (err != NULL) {
+        return err;
+    }
+
+    cmd->server->max_busy_keep_alive = atoi(arg);
+    return NULL;
+}
+
 static const char *set_pidfile(cmd_parms *cmd, void *dummy, char *arg) 
 {
     const char *err = ap_check_cmd_context(cmd, GLOBAL_ONLY);
@@ -2891,6 +2902,8 @@
   "Keep-Alive timeout duration (sec)"},
 { "MaxKeepAliveRequests", set_keep_alive_max, NULL, RSRC_CONF, TAKE1,
   "Maximum number of Keep-Alive requests per connection, or 0 for infinite" },
+{ "MaxBusyKeepAlive", set_max_busy_keep_alive, NULL, RSRC_CONF, TAKE1,
+  "Number of busy server processes upto which persistents of connections is honored, or 0
for infinite" },
 { "KeepAlive", set_keep_alive, NULL, RSRC_CONF, TAKE1,
   "Whether persistent connections should be On or Off" },
 { "IdentityCheck", set_idcheck, NULL, RSRC_CONF|ACCESS_CONF, FLAG,
diff -wurdX nodiff.pats apache-1.3.dev/src/main/http_main.c apache-1.3.dev+MaxBusyKeepAlive/src/main/http_main.c
--- apache-1.3.dev/src/main/http_main.c	Sat Aug  5 15:50:51 2000
+++ apache-1.3.dev+MaxBusyKeepAlive/src/main/http_main.c	Sun Aug  6 18:13:58 2000
@@ -2303,6 +2303,21 @@
     return old_status;
 }
 
+int ap_busy_servers(void)
+{
+    int i;
+    int busy = 0;
+    
+    ap_sync_scoreboard_image();
+    for (i = 0;  i < max_daemons_limit;  ++i) {
+	if (ap_scoreboard_image->servers[i].status >= SERVER_BUSY_READ) {
+	    ++busy;
+	}
+    }
+
+    return busy;
+}
+
 static void update_scoreboard_global(void)
 {
 #ifdef SCOREBOARD_FILE
diff -wurdX nodiff.pats apache-1.3.dev/src/main/http_protocol.c apache-1.3.dev+MaxBusyKeepAlive/src/main/http_protocol.c
--- apache-1.3.dev/src/main/http_protocol.c	Sat Feb 19 21:14:47 2000
+++ apache-1.3.dev+MaxBusyKeepAlive/src/main/http_protocol.c	Sun Aug  6 18:15:42 2000
@@ -357,6 +357,7 @@
      *   and the server configuration enables keep-alive;
      *   and the server configuration has a reasonable inter-request timeout;
      *   and there is no maximum # requests or the max hasn't been reached;
+     *   and there is no limit on busy servers or it hasn't been reached;
      *   and the response status does not require a close;
      *   and the response generator has not already indicated close;
      *   and the client did not request non-persistence (Connection: close);
@@ -382,6 +383,8 @@
         (r->server->keep_alive_timeout > 0) &&
         ((r->server->keep_alive_max == 0) ||
          (r->server->keep_alive_max > r->connection->keepalives)) &&
+	((r->server->max_busy_keep_alive == 0) ||
+	 (r->server->max_busy_keep_alive >= ap_busy_servers())) &&
         !ap_status_drops_connection(r->status) &&
         !wimpy &&
         !ap_find_token(r->pool, conn, "close") &&
diff -wurdX nodiff.pats apache-1.3.dev/src/modules/standard/mod_info.c apache-1.3.dev+MaxBusyKeepAlive/src/modules/standard/mod_info.c
--- apache-1.3.dev/src/modules/standard/mod_info.c	Thu Jun  1 19:42:28 2000
+++ apache-1.3.dev+MaxBusyKeepAlive/src/modules/standard/mod_info.c	Sun Aug  6 18:18:40 2000
@@ -421,9 +421,11 @@
                         "<tt>start: %d &nbsp;&nbsp; "
                         "min idle: %d &nbsp;&nbsp; "
                         "max idle: %d &nbsp;&nbsp; "
-                        "max: %d</tt><br>\n",
+                        "max: %d &nbsp;&nbsp; "
+                        "max busy for keep-alive: %d</tt><br>\n",
                         ap_daemons_to_start, ap_daemons_min_free,
-                        ap_daemons_max_free, ap_daemons_limit);
+                        ap_daemons_max_free, ap_daemons_limit,
+                        serv->limit_busy_keep_alive);
             ap_rprintf(r, "<strong>Max Requests:</strong> "
                         "<tt>per child: %d &nbsp;&nbsp; "
                         "keep alive: %s &nbsp;&nbsp; "




Mime
View raw message