hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jared Jacobs <ja...@kaching.com>
Subject Re: Poor performance of pooled connections worse in 4.0
Date Thu, 05 Nov 2009 22:24:24 GMT
Thanks for the response, Oleg.

Have you increased the max limit on connections per host, which is set to 2
> per default?


Yes, I did increase the limit. Here's how I initialized the HttpClient:

  private static HttpClient newMultiThreadedHttpClient() {
    return new DefaultHttpClient(
        new ThreadSafeClientConnManager(
            new BasicHttpParams()
              .setParameter(STALE_CONNECTION_CHECK, false)
              .setParameter(MAX_TOTAL_CONNECTIONS, 10)
              .setParameter(MAX_CONNECTIONS_PER_ROUTE, new ConnPerRoute() {
                public int getMaxForRoute(HttpRoute route) {
                  return 10;
                }}),
            createSchemeRegistry()),
        null);
  }

Regardless of what the max connections per host limit is set to, though, at
least the first request should not block at all. Notice that the
*minimum* elapsed
time of the N=50 requests done in each of my trial runs are all very high
when using pooled connections. This means that even the first request is
consistently slow.

I'd be happy to send you my benchmark source code. It's a single file, 100
lines.

For now, we're content using a disposable HttpClient per request. I hope to
have time to profile and investigate the connection pooling issue further
soon.

My main reason for posting to this list was the hope that someone would be
able to contradict me, ideally with measurements of their own.

Regards,
Jared


On Thu, Nov 5, 2009 at 1:18 PM, Oleg Kalnichevski <olegk@apache.org> wrote:

> Jared Jacobs wrote:
>
>> Hi there. I'm new to the group. Just upgraded from 3.1 to 4.0 for a
>> high-traffic production server cluster and noticed a drop in performance.
>> Requests are consistently taking ~40% longer. Disabling *
>> http.connection.stalecheck* had little impact.
>>
>> While investigating the issue, I noticed that switching from a shared
>> HttpClient with a ThreadSafeClientConnManager to a new simple HttpClient
>> per
>> request cuts down minimum and average request times dramatically (over
>> 80%).
>>
>> It seems the overhead for pooling and reusing connections dwarfs the
>> overhead of establishing HTTP connections. Is this just me? Anyone else
>> seen
>> this?
>>
>> Jared
>>
>>
> Jared
>
> Have you increased the max limit on connections per host, which is set to 2
> per default? Most likely your 50 worker threads spend most of their time
> blocked waiting for one of those two connections to become available.
>
> You can see what exactly is happening with the connection pool using the
> following logging config:
>
> -Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.SimpleLog
> -Dorg.apache.commons.logging.simplelog.showdatetime=true
> -Dorg.apache.commons.logging.simplelog.log.org.apache.http.impl.conn=DEBUG
>
> Hope this helps
>
> Oleg
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message