Mailing-List: contact httpclient-users-help@hc.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "HttpClient User Discussion" <httpclient-users@hc.apache.org>
Received-SPF: pass (nike.apache.org: local policy includes SPF record at
 spf.trusted-forwarder.org)
Message-ID: <1400147769.26968.12.camel@ubuntu>
Subject: Re: Thread Contention (due to use of proxys) in HttpClient 4.3
From: Oleg Kalnichevski <olegk@apache.org>
To: HttpClient User Discussion <httpclient-users@hc.apache.org>
Date: Thu, 15 May 2014 11:56:09 +0200
In-Reply-To: 
 <CADJQLmi4bxQBMCauqEegGGGbHrwnZWiXAOyQE-52q7Nzr40uqA@mail.gmail.com>
References: 
	<CADJQLmiJek1Oh3b3sKf2EoPe3=v3moJEFSEbuauKa+L9sPRj9w@mail.gmail.com>
	 <1399895327.15315.4.camel@ubuntu>
	 <CADJQLmgLwhMuK5OHMtjQiXz53dUhBS0mSqy4U_w2f0r=pHbLaw@mail.gmail.com>
	 <1399899042.18400.8.camel@ubuntu>
	 <1399929006.44087.YahooMailNeo@web142804.mail.bf1.yahoo.com>
	 <1399981202.28534.13.camel@ubuntu>
	 <CADJQLmjpB-2-v8jRVNHh3ObgyVSA2vH7A91TW9sfhjxiiP+6sA@mail.gmail.com>
	 <1400080937.15889.4.camel@ubuntu>
	 <CADJQLmi4bxQBMCauqEegGGGbHrwnZWiXAOyQE-52q7Nzr40uqA@mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit

On Thu, 2014-05-15 at 01:10 +0100, Daniel Feist wrote:
> > I re-ran the benchmark (r1594594) on my computer and unsurprisingly (for
> > me) HC 4.3 comfortably outperformed HC 3.1.
> 
> Yes I see the same running this revision. 

Could you please find out if this difference is consistent regardless of
the CPU core number used by the system? The only viable theory that I
have at the moment is for some reason HC 4.3 does not scale well on 4+
CPU core system. I have only very moderate hardware at my disposal. I
have never had a chance to test HC performance on a big box with lots of
CPUs and therefore may have missed something.


>  The difference is you are testing:
> 
> HttpClient31 vs. HttpClients.createMinimal(mgr)
> 
> and I was testing
> 
> HttpClient31 vs.
> HttpClient.custom().setMgr(mgr).disableEverything(true).build().
> 
>
> For some reason the HttpClient built by HttpClientBuilder, even when
> everything is turned is not only slower than minimalClient (to be
> expected) but also slower than DefaultHttpClient in previous versions.
>  Not a major difference, but definitely the inverse of what of whats
> in the results you sent.
> 

Honestly, there is really no significant differences between the two I
can think of. I cannot completely rule out a possibility of some green
men from Mars randomly inserting Thread#sleep() statements but I
consider it unlikely. I'll see if I can reproduce the issue locally.


> I'm still not seeing a major issue with the contention i mentioned i'd
> seen in profiler, be an interesting experiment to test performance
> with 500 treads, both with/without proxies though.
> 

I'll give it a shot this weekend.

Cheers

Oleg

> Dan
> 
> > oleg@ubuntu:~/src/apache.org/httpcomponents/httpclient-benchmark$ gr -q
> > benchmark
> > =================================
> > HTTP agent: Apache HttpClient 3.1
> > ---------------------------------
> > 1000000 POST requests
> > ---------------------------------
> > Document URI:           http://localhost:53796/echo
> > Document Length:        2048 bytes
> >
> > Concurrency level:      50
> > Time taken for tests:   64.625 seconds
> > Complete requests:      1000000
> > Failed requests:        0
> > Content transferred:    2048000000 bytes
> > Requests per second:    15473.888 [#/sec] (mean)
> > ---------------------------------
> > =================================
> > HTTP agent: Apache HttpClient (ver: 4.3.3)
> > ---------------------------------
> > 1000000 POST requests
> > ---------------------------------
> > Document URI:           http://localhost:47900/echo
> > Document Length:        2048 bytes
> >
> > Concurrency level:      50
> > Time taken for tests:   55.837 seconds
> > Complete requests:      1000000
> > Failed requests:        0
> > Content transferred:    2048000000 bytes
> > Requests per second:    17909.271 [#/sec] (mean)
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> > For additional commands, e-mail: httpclient-users-help@hc.apache.org
> >
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org