Return-Path: Delivered-To: apmail-httpd-cvs-archive@www.apache.org Received: (qmail 12689 invoked from network); 6 Sep 2006 17:36:17 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (209.237.227.199) by minotaur.apache.org with SMTP; 6 Sep 2006 17:36:17 -0000 Received: (qmail 16382 invoked by uid 500); 6 Sep 2006 17:36:16 -0000 Delivered-To: apmail-httpd-cvs-archive@httpd.apache.org Received: (qmail 16359 invoked by uid 500); 6 Sep 2006 17:36:16 -0000 Mailing-List: contact cvs-help@httpd.apache.org; run by ezmlm Precedence: bulk Reply-To: dev@httpd.apache.org list-help: list-unsubscribe: List-Post: List-Id: Delivered-To: mailing list cvs@httpd.apache.org Received: (qmail 16342 invoked by uid 99); 6 Sep 2006 17:36:15 -0000 Received: from asf.osuosl.org (HELO asf.osuosl.org) (140.211.166.49) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 06 Sep 2006 10:36:15 -0700 X-ASF-Spam-Status: No, hits=-9.4 required=10.0 tests=ALL_TRUSTED,NO_REAL_NAME X-Spam-Check-By: apache.org Received-SPF: pass (asf.osuosl.org: local policy) Received: from [140.211.166.113] (HELO eris.apache.org) (140.211.166.113) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 06 Sep 2006 10:36:13 -0700 Received: by eris.apache.org (Postfix, from userid 65534) id 1C1C01A981D; Wed, 6 Sep 2006 10:35:53 -0700 (PDT) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r440800 - in /httpd/httpd/branches/2.2.x: CHANGES STATUS docs/manual/mod/mod_proxy.xml modules/proxy/mod_proxy.c modules/proxy/mod_proxy.h modules/proxy/mod_proxy_balancer.c Date: Wed, 06 Sep 2006 17:35:52 -0000 To: cvs@httpd.apache.org From: jim@apache.org X-Mailer: svnmailer-1.1.0 Message-Id: <20060906173553.1C1C01A981D@eris.apache.org> X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N Author: jim Date: Wed Sep 6 10:35:51 2006 New Revision: 440800 URL: http://svn.apache.org/viewvc?view=rev&rev=440800 Log: Merge r427172 from trunk: Add in a very simple balancer "set" concept, which allows for members to be assigned to a particular cluster set such that members in lower-numbered sets are checked/used before those in higher ones. Also bundled in this are some HTML cleanups for the balancer manager UI. Sorry for the mixins :) Compiles/builds clean: passes test framework as well as more normal usage tests ;) Reviewed by: jim Merge r427368, r428352, r428361, r432352 from trunk: Reset standby flags for each loop through the cluster sets Initialization change. Move to a different format to force resets as well as a common technique, in case _route is updated at some point. Update docs for proxy: 1. Put params in abc order 2. Add hot-standby example 3. Add in new features that hadn't been documented yet (lbset, ...) Reviewed by: jim Modified: httpd/httpd/branches/2.2.x/CHANGES httpd/httpd/branches/2.2.x/STATUS httpd/httpd/branches/2.2.x/docs/manual/mod/mod_proxy.xml httpd/httpd/branches/2.2.x/modules/proxy/mod_proxy.c httpd/httpd/branches/2.2.x/modules/proxy/mod_proxy.h httpd/httpd/branches/2.2.x/modules/proxy/mod_proxy_balancer.c Modified: httpd/httpd/branches/2.2.x/CHANGES URL: http://svn.apache.org/viewvc/httpd/httpd/branches/2.2.x/CHANGES?view=diff&rev=440800&r1=440799&r2=440800 ============================================================================== --- httpd/httpd/branches/2.2.x/CHANGES [utf-8] (original) +++ httpd/httpd/branches/2.2.x/CHANGES [utf-8] Wed Sep 6 10:35:51 2006 @@ -1,6 +1,10 @@ -*- coding: utf-8 -*- Changes with Apache 2.2.4 + *) mod_proxy_balancer: Workers can now be defined as part of + a balancer cluster "set" in which members of a lower-numbered set + are preferred over higher numbered ones. [Jim Jagielski] + *) mod_proxy_balancer: Workers can now be defined as "hot standby" which will only be used if all other workers are unusable (eg: in error or disabled). Also, the balancer-manager displays the election Modified: httpd/httpd/branches/2.2.x/STATUS URL: http://svn.apache.org/viewvc/httpd/httpd/branches/2.2.x/STATUS?view=diff&rev=440800&r1=440799&r2=440800 ============================================================================== --- httpd/httpd/branches/2.2.x/STATUS (original) +++ httpd/httpd/branches/2.2.x/STATUS Wed Sep 6 10:35:51 2006 @@ -86,18 +86,6 @@ http://people.apache.org/~wrowe/mod_isapi-416293-to-httpd-2.2.patch +1 wrowe , fielding, jerenkrantz - * mod_proxy_balancer: Add in load balancer cluster set capability - (with docs) - Trunk version of patch: - http://svn.apache.org/viewvc?view=rev&revision=427172 - http://svn.apache.org/viewvc?view=rev&revision=427368 - http://svn.apache.org/viewvc?view=rev&revision=428352 - http://svn.apache.org/viewvc?view=rev&revision=428361 - http://svn.apache.org/viewvc?view=rev&revision=432352 (partial doc) - 2.2.x version of patch: - http://people.apache.org/~jim/patches/lbset.patch (unified from above) - +1: jim, mturk, rpluem - * mod_ext_filter: Handle filter names which include capital letters. PR 40323. Trunk patch, which applies: Modified: httpd/httpd/branches/2.2.x/docs/manual/mod/mod_proxy.xml URL: http://svn.apache.org/viewvc/httpd/httpd/branches/2.2.x/docs/manual/mod/mod_proxy.xml?view=diff&rev=440800&r1=440799&r2=440800 ============================================================================== --- httpd/httpd/branches/2.2.x/docs/manual/mod/mod_proxy.xml (original) +++ httpd/httpd/branches/2.2.x/docs/manual/mod/mod_proxy.xml Wed Sep 6 10:35:51 2006 @@ -578,6 +578,12 @@ generally 120ms), and thus prevent the firewall to drop the connection. To enable keepalive set this property value to On. + lbset + 0 + Sets the load balancer cluster set that the worker is a member + of. The load balancer will try all members of a lower numbered + lbset before trying higher numbered ones. + loadfactor 1 Worker load factor. Used with BalancerMember. @@ -608,7 +614,7 @@ status - Single letter value defining the initial status of - this worker: 'D' is disabled, 'S' is stopped + this worker: 'D' is disabled, 'S' is stopped, 'H' is hot-standby and 'E' is in an error state. Status can be set (which is the default) by prepending with '+' or cleared by prepending with '-'. Thus, a setting of 'S-E' sets this worker to Stopped and @@ -685,6 +691,22 @@ </Proxy> + +

Setting up a hot-standby, that will only be used if no other + members are available

+ + ProxyPass / balancer://hotcluster/
+ <Proxy balancer://hotcluster>
+ + BalancerMember http://1.2.3.4:8009 loadfactor=1
+ BalancerMember http://1.2.3.5:8009 loadfactor=2
+ # The below is the hot standby
+ BalancerMember http://1.2.3.6:8009 status=+H
+ ProxySet lbmethod=bytraffic + + </Proxy> + +

When used inside a Location section, the first argument is omitted and the local Modified: httpd/httpd/branches/2.2.x/modules/proxy/mod_proxy.c URL: http://svn.apache.org/viewvc/httpd/httpd/branches/2.2.x/modules/proxy/mod_proxy.c?view=diff&rev=440800&r1=440799&r2=440800 ============================================================================== --- httpd/httpd/branches/2.2.x/modules/proxy/mod_proxy.c (original) +++ httpd/httpd/branches/2.2.x/modules/proxy/mod_proxy.c Wed Sep 6 10:35:51 2006 @@ -244,6 +244,12 @@ else worker->flush_wait = ival * 1000; /* change to microseconds */ } + else if (!strcasecmp(key, "lbset")) { + ival = atoi(val); + if (ival < 0 || ival > 99) + return "lbset must be between 0 and 99"; + worker->lbset = ival; + } else { return "unknown Worker parameter"; } @@ -1800,7 +1806,7 @@ ap_rputs("\n\n" "" "" - "" + "" "\n", r); worker = (proxy_worker *)balancer->workers->elts; @@ -1819,7 +1825,8 @@ ap_rvputs(r, "", worker->s->lbfactor); - ap_rprintf(r, "", worker->s->lbset); + ap_rprintf(r, "
SchHostStatRouteRedirFAccWrRdFSetAccWrRd
", worker->s->route, NULL); ap_rvputs(r, "", worker->s->redirect, NULL); ap_rprintf(r, "%d%d", (int)(worker->s->elected)); + ap_rprintf(r, "%d%" APR_SIZE_T_FMT "", worker->s->elected); ap_rputs(apr_strfsize(worker->s->transferred, fbuf), r); ap_rputs("", r); ap_rputs(apr_strfsize(worker->s->read, fbuf), r); Modified: httpd/httpd/branches/2.2.x/modules/proxy/mod_proxy.h URL: http://svn.apache.org/viewvc/httpd/httpd/branches/2.2.x/modules/proxy/mod_proxy.h?view=diff&rev=440800&r1=440799&r2=440800 ============================================================================== --- httpd/httpd/branches/2.2.x/modules/proxy/mod_proxy.h (original) +++ httpd/httpd/branches/2.2.x/modules/proxy/mod_proxy.h Wed Sep 6 10:35:51 2006 @@ -279,6 +279,8 @@ char route[PROXY_WORKER_MAX_ROUTE_SIZ+1]; char redirect[PROXY_WORKER_MAX_ROUTE_SIZ+1]; void *context; /* general purpose storage */ + apr_size_t busy; /* busyness factor */ + int lbset; /* load balancer cluster set */ } proxy_worker_stat; /* Worker configuration */ @@ -299,29 +301,30 @@ apr_interval_time_t ttl; /* maximum amount of time in seconds a connection * may be available while exceeding the soft limit */ apr_interval_time_t timeout; /* connection timeout */ - char timeout_set; + char timeout_set; apr_interval_time_t acquire; /* acquire timeout when the maximum number of connections is exceeded */ - char acquire_set; - apr_size_t recv_buffer_size; - char recv_buffer_size_set; - apr_size_t io_buffer_size; - char io_buffer_size_set; - char keepalive; - char keepalive_set; + char acquire_set; + apr_size_t recv_buffer_size; + char recv_buffer_size_set; + apr_size_t io_buffer_size; + char io_buffer_size_set; + char keepalive; + char keepalive_set; proxy_conn_pool *cp; /* Connection pool to use */ proxy_worker_stat *s; /* Shared data */ - void *opaque; /* per scheme worker data */ - int is_address_reusable; + void *opaque; /* per scheme worker data */ + int is_address_reusable; #if APR_HAS_THREADS apr_thread_mutex_t *mutex; /* Thread lock for updating address cache */ #endif - void *context; /* general purpose storage */ + void *context; /* general purpose storage */ enum { flush_off, flush_on, flush_auto } flush_packets; /* control AJP flushing */ - int flush_wait; /* poll wait time in microseconds if flush_auto */ + int flush_wait; /* poll wait time in microseconds if flush_auto */ + int lbset; /* load balancer cluster set */ }; /* Modified: httpd/httpd/branches/2.2.x/modules/proxy/mod_proxy_balancer.c URL: http://svn.apache.org/viewvc/httpd/httpd/branches/2.2.x/modules/proxy/mod_proxy_balancer.c?view=diff&rev=440800&r1=440799&r2=440800 ============================================================================== --- httpd/httpd/branches/2.2.x/modules/proxy/mod_proxy_balancer.c (original) +++ httpd/httpd/branches/2.2.x/modules/proxy/mod_proxy_balancer.c Wed Sep 6 10:35:51 2006 @@ -93,6 +93,7 @@ /* Set to the original configuration */ workers[i].s->lbstatus = workers[i].s->lbfactor = (workers[i].lbfactor ? workers[i].lbfactor : 1); + workers[i].s->lbset = workers[i].lbset; } /* Set default number of attempts to the number of * workers. @@ -172,10 +173,12 @@ const char *route, request_rec *r) { int i; - int checking_standby = 0; - int checked_standby = 0; + int checking_standby; + int checked_standby; proxy_worker *worker; + + checking_standby = checked_standby = 0; while (!checked_standby) { worker = (proxy_worker *)balancer->workers->elts; for (i = 0; i < balancer->workers->nelts; i++, worker++) { @@ -601,6 +604,12 @@ else if (!strcasecmp(val, "Enable")) wsel->s->status &= ~PROXY_WORKER_DISABLED; } + if ((val = apr_table_get(params, "ls"))) { + int ival = atoi(val); + if (ival >= 0 && ival <= 99) { + wsel->s->lbset = ival; + } + } } if (apr_table_get(params, "xml")) { @@ -663,13 +672,13 @@ ap_rputs("\n\n" "" "" - "" + "" "" "\n", r); worker = (proxy_worker *)balancer->workers->elts; for (n = 0; n < balancer->workers->nelts; n++) { - + char fbuf[50]; ap_rvputs(r, "\n", NULL); ap_rvputs(r, "", worker->s->lbfactor); + ap_rprintf(r, "", r); - ap_rprintf(r, "", worker->s->elected); - ap_rprintf(r, "", worker->s->transferred); - ap_rprintf(r, "", worker->s->read); - ap_rputs("\n", r); + ap_rprintf(r, "\n", r); ++worker; } @@ -708,20 +719,22 @@ ap_rvputs(r, "uri, "\">\n
", NULL); ap_rputs("
Worker URLRouteRouteRedirFactorStatusFactorSetStatusElectedToFrom
uri, "?b=", balancer->name + sizeof("balancer://") - 1, "&w=", ap_escape_uri(r->pool, worker->name), @@ -677,7 +686,8 @@ ap_rvputs(r, worker->name, "", worker->s->route, NULL); ap_rvputs(r, "", worker->s->redirect, NULL); - ap_rprintf(r, "%d", worker->s->lbfactor); + ap_rprintf(r, "%d%d", worker->s->lbset); if (worker->s->status & PROXY_WORKER_DISABLED) ap_rputs("Dis ", r); if (worker->s->status & PROXY_WORKER_IN_ERROR) @@ -691,10 +701,11 @@ if (!PROXY_WORKER_IS_INITIALIZED(worker)) ap_rputs("-", r); ap_rputs("%" APR_SIZE_T_FMT "%" APR_OFF_T_FMT "%" APR_OFF_T_FMT "
%" APR_SIZE_T_FMT "", worker->s->elected); + ap_rputs(apr_strfsize(worker->s->transferred, fbuf), r); + ap_rputs("", r); + ap_rputs(apr_strfsize(worker->s->read, fbuf), r); + ap_rputs("
\n", wsel->s->lbfactor); + ap_rprintf(r, "value=\"%d\">\n", wsel->s->lbfactor); + ap_rputs("\n", wsel->s->lbset); ap_rputs("\n", r); + ap_rputs("\">\n", r); ap_rputs("\n", r); + ap_rputs("\">\n", r); ap_rputs("\n", r); + ap_rputs(">\n", r); ap_rputs("\n", r); ap_rvputs(r, "
Load factor:
LB Set:
Route:route, NULL); - ap_rputs("\">
Route Redirect:redirect, NULL); - ap_rputs("\">
Status:Disabled: s->status & PROXY_WORKER_DISABLED) ap_rputs(" checked", r); ap_rputs("> | Enabled: s->status & PROXY_WORKER_DISABLED)) ap_rputs(" checked", r); - ap_rputs(">
\npool, wsel->name), "\">\n", NULL); @@ -844,39 +857,51 @@ int total_factor = 0; proxy_worker *worker; proxy_worker *mycandidate = NULL; - int checking_standby = 0; - int checked_standby = 0; + int cur_lbset = 0; + int max_lbset = 0; + int checking_standby; + int checked_standby; ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, r->server, "proxy: Entering byrequests for BALANCER (%s)", balancer->name); /* First try to see if we have available candidate */ - while (!mycandidate && !checked_standby) { - worker = (proxy_worker *)balancer->workers->elts; - for (i = 0; i < balancer->workers->nelts; i++, worker++) { - if ( (checking_standby ? !PROXY_WORKER_IS_STANDBY(worker) : PROXY_WORKER_IS_STANDBY(worker)) ) - continue; - /* If the worker is in error state run - * retry on that worker. It will be marked as - * operational if the retry timeout is elapsed. - * The worker might still be unusable, but we try - * anyway. - */ - if (!PROXY_WORKER_IS_USABLE(worker)) - ap_proxy_retry_worker("BALANCER", worker, r->server); - /* Take into calculation only the workers that are - * not in error state or not disabled. - */ - if (PROXY_WORKER_IS_USABLE(worker)) { - worker->s->lbstatus += worker->s->lbfactor; - total_factor += worker->s->lbfactor; - if (!mycandidate || worker->s->lbstatus > mycandidate->s->lbstatus) - mycandidate = worker; + do { + checking_standby = checked_standby = 0; + while (!mycandidate && !checked_standby) { + worker = (proxy_worker *)balancer->workers->elts; + for (i = 0; i < balancer->workers->nelts; i++, worker++) { + if (!checking_standby) { /* first time through */ + if (worker->s->lbset > max_lbset) + max_lbset = worker->s->lbset; + } + if (worker->s->lbset > cur_lbset) + continue; + if ( (checking_standby ? !PROXY_WORKER_IS_STANDBY(worker) : PROXY_WORKER_IS_STANDBY(worker)) ) + continue; + /* If the worker is in error state run + * retry on that worker. It will be marked as + * operational if the retry timeout is elapsed. + * The worker might still be unusable, but we try + * anyway. + */ + if (!PROXY_WORKER_IS_USABLE(worker)) + ap_proxy_retry_worker("BALANCER", worker, r->server); + /* Take into calculation only the workers that are + * not in error state or not disabled. + */ + if (PROXY_WORKER_IS_USABLE(worker)) { + worker->s->lbstatus += worker->s->lbfactor; + total_factor += worker->s->lbfactor; + if (!mycandidate || worker->s->lbstatus > mycandidate->s->lbstatus) + mycandidate = worker; + } } + checked_standby = checking_standby++; } - checked_standby = checking_standby++; - } + cur_lbset++; + } while (cur_lbset <= max_lbset && !mycandidate); if (mycandidate) { mycandidate->s->lbstatus -= total_factor; @@ -910,42 +935,54 @@ apr_off_t mytraffic = 0; apr_off_t curmin = 0; proxy_worker *worker; - int checking_standby = 0; - int checked_standby = 0; proxy_worker *mycandidate = NULL; + int cur_lbset = 0; + int max_lbset = 0; + int checking_standby; + int checked_standby; ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, r->server, "proxy: Entering bytraffic for BALANCER (%s)", balancer->name); /* First try to see if we have available candidate */ - while (!mycandidate && !checked_standby) { - worker = (proxy_worker *)balancer->workers->elts; - for (i = 0; i < balancer->workers->nelts; i++, worker++) { - if ( (checking_standby ? !PROXY_WORKER_IS_STANDBY(worker) : PROXY_WORKER_IS_STANDBY(worker)) ) - continue; - /* If the worker is in error state run - * retry on that worker. It will be marked as - * operational if the retry timeout is elapsed. - * The worker might still be unusable, but we try - * anyway. - */ - if (!PROXY_WORKER_IS_USABLE(worker)) - ap_proxy_retry_worker("BALANCER", worker, r->server); - /* Take into calculation only the workers that are - * not in error state or not disabled. - */ - if (PROXY_WORKER_IS_USABLE(worker)) { - mytraffic = (worker->s->transferred/worker->s->lbfactor) + - (worker->s->read/worker->s->lbfactor); - if (!mycandidate || mytraffic < curmin) { - mycandidate = worker; - curmin = mytraffic; + do { + checking_standby = checked_standby = 0; + while (!mycandidate && !checked_standby) { + worker = (proxy_worker *)balancer->workers->elts; + for (i = 0; i < balancer->workers->nelts; i++, worker++) { + if (!checking_standby) { /* first time through */ + if (worker->s->lbset > max_lbset) + max_lbset = worker->s->lbset; + } + if (worker->s->lbset > cur_lbset) + continue; + if ( (checking_standby ? !PROXY_WORKER_IS_STANDBY(worker) : PROXY_WORKER_IS_STANDBY(worker)) ) + continue; + /* If the worker is in error state run + * retry on that worker. It will be marked as + * operational if the retry timeout is elapsed. + * The worker might still be unusable, but we try + * anyway. + */ + if (!PROXY_WORKER_IS_USABLE(worker)) + ap_proxy_retry_worker("BALANCER", worker, r->server); + /* Take into calculation only the workers that are + * not in error state or not disabled. + */ + if (PROXY_WORKER_IS_USABLE(worker)) { + mytraffic = (worker->s->transferred/worker->s->lbfactor) + + (worker->s->read/worker->s->lbfactor); + if (!mycandidate || mytraffic < curmin) { + mycandidate = worker; + curmin = mytraffic; + } } } + checked_standby = checking_standby++; } - checked_standby = checking_standby++; - } + cur_lbset++; + } while (cur_lbset <= max_lbset && !mycandidate); if (mycandidate) { mycandidate->s->elected++;