hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Feist <dfe...@gmail.com>
Subject Re: Thread Contention (due to use of proxys) in HttpClient 4.3
Date Wed, 21 May 2014 14:40:34 GMT
Here are the results!!  Interesting...

I reran everything:
- Added JVM options to even out garbage collection.
- Ran each test for minutes 5 minutes.
- Used Jetty SelectChannelConnector, same as you always used.  (other
one is faster, but less stable).

https://docs.google.com/a/mulesoft.com/spreadsheets/d/1j2TCeAQmrkWLZyQfetzmtqmUDaGQBiDAr20hC9gEfYE/edit#gid=590558615

Configurations of httpClient are as you have in svn accept for 4.3.3
(not minimal) which is configured as follows:

The non-minimal configuration for 4.3.3 is:
final RequestConfig requestConfig = RequestConfig.custom()
        .setStaleConnectionCheckEnabled(false)
        .setExpectContinueEnabled(false)
        .build();
this.httpclient = HttpClients.custom()
        .setConnectionManager(this.mgr)
        .disableContentCompression()
        .setDefaultRequestConfig(requestConfig)
        .build();


Dan

On Sat, May 17, 2014 at 8:15 PM, Oleg Kalnichevski <olegk@apache.org> wrote:
> On Sat, 2014-05-17 at 00:13 +0100, Daniel Feist wrote:
>> > Could you please find out if this difference is consistent regardless of
>> > the CPU core number used by the system?
>>
>> Done, see other email I didn't record all figures, but did run
>> multiple times and records differences.
>>
>> > Honestly, there is really no significant differences between the two I
>> > can think of. I cannot completely rule out a possibility of some green
>> > men from Mars randomly inserting Thread#sleep() statements but I
>> > consider it unlikely. I'll see if I can reproduce the issue locally.
>>
>> I couldn't imagine a difference but in testing on 32-core with
>> BlockingChannelConnector I got:
>>
>> - 3.1                                               |   75 K TPS
>> - 4.3.3 Minimal / 4.2.6                 |   74 K TPS
>> - 4.3.3 DefultHttpClient               |   72 K TPS
>> - 4.3.3 Builder (from your test)   |   68 K TPS
>>
>> The difference is small, but it's an interesting problem, working out
>> whats going on.  74 vs 75 is irrelevant, but 68 vs. 75 is bigger.
>>
>> >> I'm still not seeing a major issue with the contention i mentioned i'd
>> >> seen in profiler, be an interesting experiment to test performance
>> >> with 500 treads, both with/without proxies though.
>> >>
>> >
>> > I'll give it a shot this weekend.
>>
>> Not sure you'll see any difference is cpu is constrained. If you do do
>> it, I can test on 32-core box for you.
>>
>
> Daniel
>
> I removed all dynamic proxies and got approximately 10% better
> performance with my two CPU core PC.
>
> Could you please build HttpClient from my private experimental branch at
> github and re-run your tests?
>
> https://github.com/ok2c/httpclient/tree/dyn_proxies
> https://github.com/ok2c/httpclient/commit/246095522504ddffe07caf70db4c051fa0d31872
>
> Cheers
>
> Oleg
>

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org


Mime
View raw message