Mailing-List: contact httpclient-users-help@hc.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "HttpClient User Discussion" <httpclient-users@hc.apache.org>
Received-SPF: pass (athena.apache.org: domain of dfeist@gmail.com designates
 209.85.215.54 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <1400354144.25068.7.camel@ubuntu>
References: 
 <CADJQLmiJek1Oh3b3sKf2EoPe3=v3moJEFSEbuauKa+L9sPRj9w@mail.gmail.com>
	<1399895327.15315.4.camel@ubuntu>
	<CADJQLmgLwhMuK5OHMtjQiXz53dUhBS0mSqy4U_w2f0r=pHbLaw@mail.gmail.com>
	<1399899042.18400.8.camel@ubuntu>
	<1399929006.44087.YahooMailNeo@web142804.mail.bf1.yahoo.com>
	<1399981202.28534.13.camel@ubuntu>
	<CADJQLmjpB-2-v8jRVNHh3ObgyVSA2vH7A91TW9sfhjxiiP+6sA@mail.gmail.com>
	<1400080937.15889.4.camel@ubuntu>
	<CADJQLmi4bxQBMCauqEegGGGbHrwnZWiXAOyQE-52q7Nzr40uqA@mail.gmail.com>
	<1400147769.26968.12.camel@ubuntu>
	<CADJQLmgyyuBV2M7GUZCHz0+GHevSMa3ussCKEawb2uaQr=Q=HA@mail.gmail.com>
	<1400354144.25068.7.camel@ubuntu>
Date: Wed, 21 May 2014 15:40:34 +0100
Message-ID: 
 <CADJQLmiuSWS2AEqmOdqroFUbBa+d06KEwccxjkAmpLWR3vAm0w@mail.gmail.com>
Subject: Re: Thread Contention (due to use of proxys) in HttpClient 4.3
From: Daniel Feist <dfeist@gmail.com>
To: Oleg Kalnichevski <olegk@apache.org>
Cc: HttpClient User Discussion <httpclient-users@hc.apache.org>
Content-Type: text/plain; charset=UTF-8

Here are the results!!  Interesting...

I reran everything:
- Added JVM options to even out garbage collection.
- Ran each test for minutes 5 minutes.
- Used Jetty SelectChannelConnector, same as you always used.  (other
one is faster, but less stable).

https://docs.google.com/a/mulesoft.com/spreadsheets/d/1j2TCeAQmrkWLZyQfetzmtqmUDaGQBiDAr20hC9gEfYE/edit#gid=590558615

Configurations of httpClient are as you have in svn accept for 4.3.3
(not minimal) which is configured as follows:

The non-minimal configuration for 4.3.3 is:
final RequestConfig requestConfig = RequestConfig.custom()
        .setStaleConnectionCheckEnabled(false)
        .setExpectContinueEnabled(false)
        .build();
this.httpclient = HttpClients.custom()
        .setConnectionManager(this.mgr)
        .disableContentCompression()
        .setDefaultRequestConfig(requestConfig)
        .build();


Dan

On Sat, May 17, 2014 at 8:15 PM, Oleg Kalnichevski <olegk@apache.org> wrote:
> On Sat, 2014-05-17 at 00:13 +0100, Daniel Feist wrote:
>> > Could you please find out if this difference is consistent regardless of
>> > the CPU core number used by the system?
>>
>> Done, see other email I didn't record all figures, but did run
>> multiple times and records differences.
>>
>> > Honestly, there is really no significant differences between the two I
>> > can think of. I cannot completely rule out a possibility of some green
>> > men from Mars randomly inserting Thread#sleep() statements but I
>> > consider it unlikely. I'll see if I can reproduce the issue locally.
>>
>> I couldn't imagine a difference but in testing on 32-core with
>> BlockingChannelConnector I got:
>>
>> - 3.1                                               |   75 K TPS
>> - 4.3.3 Minimal / 4.2.6                 |   74 K TPS
>> - 4.3.3 DefultHttpClient               |   72 K TPS
>> - 4.3.3 Builder (from your test)   |   68 K TPS
>>
>> The difference is small, but it's an interesting problem, working out
>> whats going on.  74 vs 75 is irrelevant, but 68 vs. 75 is bigger.
>>
>> >> I'm still not seeing a major issue with the contention i mentioned i'd
>> >> seen in profiler, be an interesting experiment to test performance
>> >> with 500 treads, both with/without proxies though.
>> >>
>> >
>> > I'll give it a shot this weekend.
>>
>> Not sure you'll see any difference is cpu is constrained. If you do do
>> it, I can test on 32-core box for you.
>>
>
> Daniel
>
> I removed all dynamic proxies and got approximately 10% better
> performance with my two CPU core PC.
>
> Could you please build HttpClient from my private experimental branch at
> github and re-run your tests?
>
> https://github.com/ok2c/httpclient/tree/dyn_proxies
> https://github.com/ok2c/httpclient/commit/246095522504ddffe07caf70db4c051fa0d31872
>
> Cheers
>
> Oleg
>

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org