httpd-bugs mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject DO NOT REPLY [Bug 44855] New: irregular balancing with sticky-session after re-enabling worker or lbset-changes
Date Tue, 22 Apr 2008 17:51:43 GMT

           Summary: irregular balancing with sticky-session after re-
                    enabling worker or lbset-changes
           Product: Apache httpd-2
           Version: 2.2-HEAD
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: normal
          Priority: P2
         Component: mod_proxy_balancer


I've a little issue to discuss regarding sticky-sessions, which could result in
very high loads on the backend-servers.

After changing the lbset of a worker to a lower one (eg. from 1 to 0), it will
receive a large number of requests. This is because all requests (with and
without sticky-session) are counted the same way.
This would also happen if a worker left an unusable-state (not tested, but its
the same principle).

You can find an example at the end of this posting.

As already mentioned in the sourcecode (file: mod_proxy_balancer.c, function:
proxy_balancer_pre_request), this is an open issue:
             * TODO: Abstract the below, since this is dependent
             *       on the LB implementation

Possible solutions for the sticky-session issue:

1) Configurable counting of lbstatus
   - New proxy-balancer option: lbmethod_counting=[all|new|(sticky?)]
     all: the current behaviour (default)
     new: only new sessions will modify the lbstatus
     sticky: only sticky session will modify the lbstatus <- useful?

  Changes needed:
     mod_proxy_balancer.c (proxy_balancer_pre_request, find_best_by* <- only
for 'sticky')
     mod_proxy.h (struct proxy_balancer)
     and a few other files to read the new config-parameter

2) Implementation of a "soft fade-in". After changes to lbset, error-states,
... a soft fade-in-flag will be set.
   For a defined period of time, the counting of lbstatus will be reduced to
new sessions.
   - New proxy-balancer option: lbmethod_fadeintime=<seconds int>
     0 : the current behaviour (default)
     >0: fadeintime

   Changes needed:
     mod_proxy_balancer.c (proxy_balancer_pre_request)
     mod_proxy.h (struct proxy_balancer, struct proxy_worker_stat:+ apr_time_t
     all places where lbset and error-states are modified
     and a few other files to read the new config-parameter

My current favorite is solution 1), because there is less code to modify and it
would meet my requirements.

While digging around in the sourcecode I've also found a little flaw within
that TODO-Section:
lbsets aren't handled at all. So the lbstatus for all workers in all lbsets are
modified, which would result in unbeautiful values of lbstatus (e.g. -1.000.000
: -1.000.000 : +2.000.000)
This little patch could fix that flaw:
             * TODO: Abstract the below, since this is dependent
             *       on the LB implementation
           -  if (PROXY_WORKER_IS_USABLE(workers)) {
           + if (PROXY_WORKER_IS_USABLE(workers) && workers->s->lbset ==
runtime->s->lbset) {

What do you think about it?

Ciao - Marcus

- 3 workers with sticky-session (a, b, c)
- 1 new session per second with 50(+1) requests over 300 seconds
- initial lbsets: a:0, b:0, c:1

After changing the lbset of worker c to 0, the lbstatus will be cleared
(recalc_factors()). So we start at 0.

// all new session will go to c
Second  :    0
ReqDone : a=  0.00, b=  0.00, c=   0.00
lbstatus: a=  0.0, b= -0.0, c=  0.0
Sessions: a= 150.00, b= 150.00, c=   0.00

Second  :   60
ReqDone : a= 1378.58, b= 1367.42, c= 365.00
lbstatus: a= -1024.7, b= -991.3, c= 2016.0
Sessions: a= 120.50, b= 119.50, c=  60.00

Second  :  180
ReqDone : a= 3183.58, b= 3152.42, c= 2895.00
lbstatus: a= -319.7, b= -226.2, c= 546.0
Sessions: a=  60.50, b=  59.50, c= 180.00

// since now all new session will go to a + b
// but we are also losing old-session
Second  :  192 
ReqDone : a= 3298.08, b= 3264.92, c= 3280.00
lbstatus: a= -51.2, b= 48.2, c=  3.0
Sessions: a=  54.50, b=  53.50, c= 192.00

// reached break-even point: c loses sessions
Second  :  301
ReqDone : a= 4316.83, b= 4317.33, c= 6767.83
lbstatus: a= 2451.5, b= 2450.0, c= -4901.5
Sessions: a=  54.00, b=  55.00, c= 191.00

// till c has 0 active sessions
Second  :  575
ReqDone : a= 9792.00, b= 9792.00, c= 9792.00
lbstatus: a=  0.0, b= -0.0, c=  0.0
Sessions: a= 150.00, b= 150.00, c=   0.00

// now we have a similar situation to second 0

Configure bugmail:
------- You are receiving this mail because: -------
You are the assignee for the bug.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message