httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Leif Hedstrom <zw...@apache.org>
Subject Re: Proposal/RFC: "informed" load balancing
Date Sat, 02 May 2015 21:28:05 GMT
I’m Cc: this to dev@trafficserver.apache.org, since I think this is something some of our
dev would be interested in. There are a few other replies to this thread already, which can
be seen on the archives.

As has been mentioned in another reply, I think the header name ought to be something ^X-
(see RFC 6648). Backend-Info or Backend-Capacity-Info or some such?

Cheers,

— leif



> On Apr 29, 2015, at 10:54 PM, Jim Riggs <apache-lists@riggs.me> wrote:
> 
> [ Long message and proposal follows. Bear with me. There are a lot of words, but that
is because we need a lot of help/input! ;-) ]
> 
> So, this has come up in the past several times, and we discussed it again this year at
ApacheCon: How do we get the load balancer to make smarter, more informed decisions about
where to send traffic?
> 
> The different LB methods provide some different attempts at balancing traffic, but ultimately
none of them is "smart" about its decision. Other than a member being in error state, the
balancer makes its decision solely based on configuration (LB set, factor, etc.) and its own
knowledge of the member (e.g. requests, bytes). What we have often discussed is a way to get
some type of health/load/capacity information from the backend to make informed balancing
decisions.
> 
> One method is to use health checks (a la haproxy, AWS ELBs, etc.) that request one or
more URLs and the response code/time indicates whether or not the service is up and available,
allowing more proactive decisions. While this is better than our current state of reactively
marking members in error state based on failed requests, it still provides a limited view
of the health/state of the backend.
> 
> We have also discussed implementing a way for backends to communicate a magical "load"
number to the front end to take into account as it balances traffic. This would give a much
better view into the backend's state, but requires some way to come up with this calculation
that each backend system/server/service/app must provide. This then has to be implemented
in all the various backends (e.g. httpd, tomcat, php-fpm, unicorn, mongrel, etc., etc.), probably
a hard sell to all of those projects. And, the front end would have limited control over what
that number means or how to use it.
> 
> During JimJag's balancer talk at ApacheCon this year, he brought up this issue of "better,
more informed" decision making again. I put some thought into it that night and came up with
some ideas. Jim, Covener, Trawick, Ruggeri, and I then spent some time over the next couple
of days talking it through and fleshing out some of the details.
> 
> Based on all of that, below is what I am proposing. I have some initial code that I am
working on to implement the different pieces of this, and I will put them up in bugz or somewhere
when they're a little less rudimentary.
> 
> --
> 
> Our hope is to create a general standard that can be used by various projects, products,
proxies, servers, etc., to have a more consistent way for a load balancer to request and receive
useful internal state information from its backend nodes. This information can then be used
by the *frontend* software/admin (this is the main change from what we have discussed before)
to calculate a load factor appropriate for each backend node.
> 
> This communication uses a new, standard HTTP header, "X-Backend-Info", that takes this
form in RFC2616 BNF:
> 
>    backend-info  = "version" "=" numeric-entry
>                    [
>                      *LWS "," *LWS
>                      #( numeric-entry | string-entry )
>                    ]
> 
>    numeric-entry = numeric-field "=" ( float | <"> float <"> )
>                    ; that is, numbers may optionally be enclosed in
>                    ; quotation marks
> 
>    float         = 1*DIGIT [ "." 1*DIGIT ]
> 
>    numeric-field = "workers-max"
>                    ; maximum number of workers the backend supports
>                  | "workers-used"
>                    ; current number of used/busy workers
>                  | "workers-allocated"
>                    ; current number of allocated/ready workers
>                  | "workers-free"
>                    ; current number of workers available for use
>                    ; (generally the difference between workers-max and
>                    ; workers-used, though some implementations may have
>                    ; a different notion)
>                  | "uptime"
>                    ; number of seconds the backend has been running
>                  | "requests"
>                    ; number of requests the backend has processed
>                  | "memory-max"
>                    ; total amount of memory available in bytes
>                  | "memory-used"
>                    ; amount of used memory in bytes
>                  | "memory-allocated"
>                    ; amount of allocated/committed memory in bytes
>                  | "memory-free"
>                    ; amount of memory available for use (generally
>                    ; the difference between memory-max and memory-used,
>                    ; though some implementations may have a different
>                    ; notion)
>                  | "load-current"
>                    ; the (subjective) current load for the backend
>                  | "load-5"
>                    ; the (subjective) 5-minute load for the backend
>                  | "load-15"
>                    ; the (subjective) 15-minute load for the backend
> 
>    string-entry  = string-field "=" ( token | quoted-string )
> 
>    string-field  = "provider"
>                    ; informational description of backend information
>                    ; provider (module, container, subsystem, app, etc.)
> 
> 
> As used here, "worker" is an overloaded term whose precise meaning is backend-dependent.
It might refer to processes, threads, pipelines, or whatever the backend system/server/service/app
uses to measure or limit its number of active, processing connections.
> 
> The process-flow looks like this:
> 
> 1. The frontend (periodically based on time or requests, or on demand) as part of either
(1) a normal proxied request or (2) a dedicated health check adds an "X-Backend-Info" request
header to a backend request, informing the backend that it wants node state information. I.e.:
> 
>    X-Backend-Info: version=1.0
> 
> 2. The backend node receives a request with an "X-Backend-Info" header specifying a version
it supports.
> 
> 3. A supporting backend node SHOULD insert one or more "X-Backend-Info" response headers
with any subset of the backend-info fields that it supports, including the required "version"
field. The version of information provided MUST be less than or equal to the version requested.
(The fields are standardized so that various frontends know what to expect, rather than each
backend system/server/service/app creating its own fields/values.) E.g.:
> 
>    X-Backend-Info: version=1.0, provider="Backend X", workers-max=1000,
>                    workers-used=517, workers-free=483, uptime=19234,
>                    requests=85939
> 
> 4. The backend MUST add the "X-Backend-Info" token to the "Connection" response header,
making it a hop-by-hop field that is removed by the frontend from the downstream response
(RFC2616 14.10 and RFC7230 6.1). [Note there appears to be an httpd bug here that I intend
to submit and that needs to be addressed.]
> 
>    Connection: X-Backend-Info
> 
> 5. The frontend parses the backend-info entries in the received "X-Backend-Info" response
header. The values are then used as part of either an internal or an administrator-specified
calculation to determine the load factor or weight of that node for subsequent requests.
> 
> 6. The frontend MUST remove the "X-Backend-Info" hop-to-hop response header per RFCs.
> 
> --
> 
> As for httpd implementation, this has two pieces. The first is when httpd is used as
a backend node behind a load balancer and must provide X-Backend-Info response data. For this,
I have created a module tentatively named mod_proxy_backend_info that does nothing except
insert an output filter to populate the response header with version, provider, workers-*,
request, uptime, and load-* values when the request header is present. Here is an example
request-response:
> 
> % curl -v -H 'X-Backend-Info: version=1.0' http://localhost/
> *   Trying 127.0.0.1...
> * Connected to localhost (127.0.0.1) port 80 (#0)
>> GET / HTTP/1.1
>> User-Agent: curl/7.41.0
>> Host: localhost
>> Accept: */*
>> X-Backend-Info: version=1.0
>> 
> < HTTP/1.1 200 OK
> < Date: Thu, 30 Apr 2015 04:32:08 GMT
> < Server: Apache/2.4.9 (Unix) PHP/5.5.14
> < Last-Modified: Wed, 15 Apr 2015 14:04:54 GMT
> < ETag: "2d-513c3d4d78d80"
> < Accept-Ranges: bytes
> < Content-Length: 45
> < X-Backend-Info: version=1.0, provider="mod_proxy_backend_info [Apache/2.4.9 (Unix)
PHP/5.5.14]", workers-max=256, workers-busy=1, workers-ready=4, workers-free=255, uptime=1448,
requests=3, load-current=1.737305, load-5=1.733887, load-15=1.668457
> < Connection: X-Backend-Info
> < Content-Type: text/html
> <
> <html><body><h1>It works!</h1></body></html>
> 
> 
> The second piece is when httpd is used as the load balancer. For this, I have created
a module tentatively named mod_lbmethod_bybackendinfo that will:
> 
> 1. Periodically (based on elapsed time, number of requests, or both since last update)
insert the X-Backend-Info request header into a proxied request.
> 
> 2. Parse and remove the X-Backend-Info response header.
> 
> 3. Calculate the member's "informed" load factor based on a formula specified by the
user/admin in the configuration. I hope to just use the existing lbfactor field to store this
calculated value. Then we can use existing logic to balance based on lbset and lbfactor for
subsequent requests.
> 
> 4. Store the current time and request count in the member's data structure so the lbmethod
knows when it needs to be updated again.
> 
> 
> What I need from all of you:
> 
> - Input/commentary on the proposed idea, approach, and implementation. Renaming things,
additional fields that might be useful, etc., are all up for discussion.
> 
> - Help with handling the configuration formula mentioned in #3 above. Can we just add
some math operators to the expression parser to handle this? What all operations/functions
might we need (+-*/? max? min? ternary if-then-else? ...)? A simple-ish example (something
like this maybe?):
> 
> <Proxy "balancer://...">
>  BalancerMember ...
>  ...
>  ProxySet \
>    lbmethod=bybackendinfo \
>    backendupdateseconds=30 \
>    backendupdaterequests=100 \
>    backendformula="%{BACKEND:uptime} -lt 120 ? 1 : %{BACKEND:workers-free} / %{BACKEND:workers-max}
* 100"
> </Proxy>
> 
> - [Near-long-term] Help adding X-Backend-Info backend support and documentation to various
projects. Tomcat, php-fpm, others(?) should be fairly easy to implement and submit patches.
This work does us no good if none of our backends support it.
> 
> - [Long-term] Help adding X-Backend-Info frontend support and documentation to various
projects to help this become an "accepted ad-hoc standard"...or something like that. Nginx,
haproxy, and many others would be targets.
> 
> 
> Warn out from writing all of this and hopeful that someone other than me actually cares,
I wish you all well today/tonight!
> 
> - Jim
> 


Mime
View raw message