tomcat-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Filip Hanik - Dev Lists <>
Subject Re: BIO performance issues
Date Wed, 04 May 2011 15:17:34 GMT
On 5/3/2011 2:02 PM, Mark Thomas wrote:
> All,
> Summary
> -------
> While trying to align the documentation for maxConnections with the
> actual implementation a couple of performance issue have been identified
> with the BIO connector. All bar one of these have been fixed. A fix
> needs to be agreed for the remaining issue, ideally before the next
> 7.0.x release.
> Background
> ----------
> New connections to the BIO connector are placed in a queue. When a
> thread is available, it takes the next connection of the queue and
> processes it. When the connection has been processed if the connection
> is kept-alive it is placed back on the queue. If there is no keep-alive
> the connection is closed.
> Scenario
> --------
> This ended up being very long, so I moved it to the end. The exact
> pattern of delays will vary depending on timeouts, request frequency
> etc. but the scenario shows an example of how delays can occur. The
> short version is that requests with data to process (particularly new
> connections) tend to get delayed in the queue waiting for a thread to
> process them when the threads are all tied up processing keep-alive
> connections.
> Root cause
> ----------
> The underlying cause of all of the performance issues observed is when
> the threads are tied up doing HTTP keep-alive when there is no data
> process but there are other connections in the queue that do have data
> that could be processed.
> Solution A
> ----------
> NIO is designed to handle this using a poller. That isn't available to
> BIO so I attempted to simulate it. That generated excessive CPU load so
> I do not think simulated polling is the tight solution.
> Solution B
> ----------
> Return to the Tomcat 6 implementation where maxConnections == maxThreads.
> Additional clean-up
> -------------------
> maxConnections is unnecessary in APR since pollerSize performs the same
> function.

I'd -1 to Solution A and Solution B, mostly because the very descriptive example below shows
a scenario that can be configured around.
I would say that
- maxThreads=200
- maxConnections=10000
- connectionTimeout=20000
- keepAliveTimeout=20000
is a gross misconfiguration for the BIO connector if you really expect this type of concurrency.
the 20 second timeout stems from our 10Mbps 
network days. I can equally prove that the old implementation has no fairness what so ever.
As some connections will stay on the backlog 
until they expire while some get accepted.

In a similar fashion, we can also craft a test run that will yield a substantial improvement
over the old implementation in throughput.
So there is a test case to prove every scenario.

Here is what I propose, and you'll see that it's pretty much inline with what you suggest.

a) add in a flag that lets keep alive be turned off during concurrency
this would be the default value. and would achieve exactly what you want in your a)

> a) restore disabling keep-alive when threads used>= 75% of maxThreads

> b) remove maxConnections and associated code from the APR connector
or you can simply make pollerSize==maxConnections
meaning that
setPollerSize(5000) is the same as setMaxConnections(5000)
that way you retain the same properties across connectors

> c) remove the configuration options for maxConnections from the BIO
> connector
I think you still misunderstand why maxConnections is there, at some point you need to push
back on the TCP stack.
and still be able to control when to accept new connections.
Keep this setting in there, exactly as it is.

> d) use maxThreads instead of maxConnections for the BIO connector
if you feel so strongly on this, simple set maxConnections=200 as the default value for BIO

so if the new implementation has these default values for BIO


this will grant you the exact behavior of the old implementation, while giving others the
benefit of configuring for better performance 
using BIO


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message