trafficserver-github mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <>
Subject [GitHub] [trafficserver] sudheerv opened a new pull request #6968: RateLimiting and Connection Config changes
Date Tue, 30 Jun 2020 20:04:11 GMT

sudheerv opened a new pull request #6968:

   Link to related talk during ATS Spring'20 Summit
   Email to dev/users@
   Re: Connection and Timeout Tuning, config name changes in 9.x
   Sudheer Vinukonda <>
   Leif Hedstrom
   Sat, May 16 at 10:41 AM
   Thanks, Bryan! Based on the discussion with Bryan on slack, below is the newer proposed
config changes: 
   1) Rename "" to ""
   2) Rename "proxy.config.http.server_max_connections" to ""
   3) "" - This config will stay put to limit the max number
of in-flight requests (concurrency) + idle (socket) connections 
   4) "" - This config will stay put to limit the max
number of open connections (FD, rlimits etc), but eventually will be deprecated and replaced
with {{ "" + "" }}
   Please provide any feedback/comments/concerns.
   On Friday, May 15, 2020, 11:11:53 AM PDT, Bryan Call <> wrote:
   After seeing the GitHub PR I think we should keep since
it is the number of socket connections (active + idle) and then rename,
as you suggested in your email, to or
(as Sudheer suggested on Slack).
   I would like to see us get rid of in the long term
and use max_connections_in/out if possible.
   > On May 12, 2020, at 5:17 PM, Sudheer Vinukonda <>
   > 9.x (re)introduces a feature (it was accidentally "lost" during some unrelated refactoring)
that will allow to configure limits on max "active" connections that can be allowed at any
given time (
   > Given that this feature is based on tracking "active connections", it should work
very well with HTTP/1.1 traffic (where, 1 active connection = 1 request). However, with HTTP/2
and stream multiplexing, this is no longer true and it isn't nearly as effective. One can
still use an average number of concurrent streams/connection to configure the limits, but,
ideally, what would work for both HTTP/1.1 and HTTP/2 (and going forward with QUIC/HTTP/3)
would be to track and (rate) limit request level concurrency. There is some ongoing work on
this and to align better with that work, we are proposing to make the below changes to existing
configs with 9.x
   > 1) Rename "" to ""
   > 2) Rename "" to ""
   > 3) - This config will stay put to limit the
max number of open connections (FD, rlimits etc)
   > More context on the new changes -
   > The idea is to tune active connections that can be handled (based on the available
resources/capacity (CPU, Memory, Network bandwidth) and minimize/remove dependency on having
to tune connection timeouts (active/inactive/keep-alive etc) which are very hard to tune.
   > The primary goal is to limit the max active connections allowed at any given instant.
The resource requirement for an idle/inactive vs active connections are completely different
- For e.g an idle connection really only consumes memory resource, while an active connection
consumes CPU, network besides memory. And allowing to tune/cap the max active connections
based on the deployed capacity for the resources available, would make timeout tuning almost
redundant and no op. Otherwise, we'd have to tune the timeouts to estimate throughput which
is very hard as it's hard to justify how large or small we want the active timeout to be or
keep alive timeout to be. For e.g in a non-peak hour, we could let the active timeout be much
higher than the default, while during peak hour, we'd want to limit it to ensure we are not
starving resources on one connection.
   > Note: there's one little TODO item here. PluginVC's connections are not tracked in
the NetVC's active queue because these are bogus internal connections. However, for some users,
internal traffic is significant (e.g sideways calls, or background fetches etc) and does consume
plenty of resources. Since these internal requests don't impact ingress network, and have
a slightly different resource requirements than client originated requests, it might be better
to track these using a separate config for internal requests. Will follow that part up with
a separate PR.

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:

View raw message