Mailing-List: contact httpclient-users-help@hc.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "HttpClient User Discussion" <httpclient-users@hc.apache.org>
Received-SPF: pass (nike.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
  s=s1024; d=yahoo.com;
  h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:MIME-Version:Content-Type:Content-Transfer-Encoding;
  b=VmF3rvN37ViuPNwFh+Ntlhj7G1IvHZKtGUVHaLxFoNxiUoRtQTpARXiNL8Fryw46kI3xZRiqnxCVTo5UEcdenECXhuaAiIB9YigMYGzR3PfaVpgdtqFndZzEkNqxPXM4F/RZBCkusEpHaTpci0aa6JpvRW6xFFg3Cys5ErGjvL4=;
Message-ID: <976640.67459.qm@web113610.mail.gq1.yahoo.com>
Date: Sat, 5 Mar 2011 00:00:43 -0800 (PST)
From: Tony Anecito <adanecito@yahoo.com>
Subject: ApacheHttp and Tomcat 7 tests...
To: HttpClient User Discussion <httpclient-users@hc.apache.org>
Cc: Tomcat Users Group <users@tomcat.apache.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable

Hi Oleg,=0A=0ASummary The following test results show me that using the loc=
alhost as the =0Aserver name and the ApacheHttp client running on the serve=
r as done with the =0AApacheHttp tests by Apache components group gets clos=
e to or bettter than =0Aexpected. Using localhost I am guessing bypasses us=
ing the network and running =0A500,000 requests over 20 threads gets a low =
average.=0A=0AI still need to have the servlet setup to send data but I am =
wondering if the =0AApache component tests used Tomcat at all and instand j=
ust read a file from =0AApache Web Server. I will look further into this al=
so to make sure I get closer =0Ato what the tests did.=0A=0ASince I think I=
 am getting closer to how Tomcat responds for a bare servlet with =0Ano cod=
e in it and maybe no network component I am more comfortable in =0Aundersta=
nding what Tomcat can truely do from a performance perspective.=0A=0A=0AIf =
I go back to my 6 core server and run the same test (client running on the =
=0Aserver where Tomcat is as the I did previously on my old laptop I get:=
=0A=0AhttpClient execute time in nanoseconds: 209391137=0AhttpClient execut=
e time in nanoseconds: 1595734=0AhttpClient execute time in nanoseconds: 12=
98489=0AhttpClient execute time in nanoseconds: 1238355=0AhttpClient execut=
e time in nanoseconds: 1262800=0AhttpClient execute time in nanoseconds: 11=
96312=0AhttpClient execute time in nanoseconds: 1253512=0AhttpClient execut=
e time in nanoseconds: 1457866=0AhttpClient execute time in nanoseconds: 12=
15867=0AhttpClient execute time in nanoseconds: 1254000=0A=A0=0AConcurrency=
 level: 1=0ATime taken for tests: 0.234 seconds=0AComplete requests: 10=0AF=
ailed requests: 0=0AContent transferred: 0 bytes=0ARequests per second: 42.=
735043 [#/sec] (mean)=0ATime per request: 23.4 [ms] (mean)=0A=0AIf I run ne=
xt with the server name set to localhost instead of a internet url as =0Ado=
ne on the Apache tests I get (This should cause the network to be bypassed =
=0Aperhaps even the TCP/IP Stack? note the much lower response times):=0A=
=0AhttpClient execute time in nanoseconds: 100221746=0AhttpClient execute t=
ime in nanoseconds: 890756=0AhttpClient execute time in nanoseconds: 688845=
=0AhttpClient execute time in nanoseconds: 688844=0AhttpClient execute time=
 in nanoseconds: 689823=0AhttpClient execute time in nanoseconds: 614044=0A=
httpClient execute time in nanoseconds: 585200=0AhttpClient execute time in=
 nanoseconds: 604266=0AhttpClient execute time in nanoseconds: 657066=0Ahtt=
pClient execute time in nanoseconds: 408223=0A=A0=0AConcurrency level: 1=0A=
Time taken for tests: 0.109 seconds=0AComplete requests: 10=0AFailed reques=
ts: 0=0AContent transferred: 0 bytes=0ARequests per second: 91.74312 [#/sec=
] (mean)=0ATime per request: 10.9 [ms] (mean)=0A=0A=0AIf I now run a 500,00=
0 request thread over 20 sockets (localhost and running the =0Aclient app o=
n the Tomcat serverI get:=0A=0AConcurrency level: 20=0ATime taken for tests=
: 14.549 seconds=0AComplete requests: 500000=0AFailed requests: 0=0AContent=
 transferred: 0 bytes=0ARequests per second: 34366.625 [#/sec] (mean)=0ATim=
e per request: 0.029098 [ms] (mean)=0A=0A=0A=0A----- Original Message ----=
=0AFrom: Tony Anecito <adanecito@yahoo.com>=0ATo: HttpClient User Discussio=
n <httpclient-users@hc.apache.org>=0ASent: Thu, March 3, 2011 10:49:53 PM=
=0ASubject: Re: Slowness of 4.1=0A=0AHi Oleg,=0A=0AHere is some numbers whe=
n running TestHttpClient4. I am running from Eclipse =0Ausing jdk 1. 6.0.22=
 and a Aspire 5670 which has Intel Core Duo T2300 (1.66 GHz, =0A667 MHz FSB=
, 2MB L2 cache) and 2GB of DDR2 memory. Note how for a single thread =0Arun=
ning 10 requests the response time starts out very slow at 150msec then get=
s =0Ato 1.43msec at the end.=0A=0AhttpClient execute time in nanoseconds:=
=A0150181377=0AhttpClient execute time in nanoseconds:=A02180445=0AhttpClie=
nt execute time in nanoseconds:=A01540419=0AhttpClient execute time in nano=
seconds:=A01757207=0AhttpClient execute time in nanoseconds:=A01759721=0Aht=
tpClient execute time in nanoseconds:=A01492368=0AhttpClient execute time i=
n nanoseconds:=A01493765=0AhttpClient execute time in nanoseconds:=A0147448=
9=0AhttpClient execute time in nanoseconds:=A01456610=0AhttpClient execute =
time in nanoseconds:=A01437613=0ADocument URI:=A0=A0xxx=0ADocument Length:=
=A00 bytes=0AConcurrency level:=A01=0ATime taken for tests:=A00.187 seconds=
=0AComplete requests:=A010=0AFailed requests:=A00=0AContent transferred:=A0=
0 bytes=0ARequests per second:=A053.475933 [#/sec] (mean)=0ATime per reques=
t:=A018.7 [ms] (mean)=0A=0A=0AWhen I increase the number of threads to 50 a=
nd requests to 500000 I get a max =0Areq of 2.2K per second and around 400m=
icroseconds.=0AThe CPU utilization is between 50-60% on the laptop and arou=
nd 5-7% on my 6 core =0A=0Aserver with Tomcat 7. Tommow I will setup the te=
st to run from a faster client =0Aand see the results.=0A=0A=0A=0A----- Ori=
ginal Message ----=0AFrom: Tony Anecito <adanecito@yahoo.com>=0ATo: HttpCli=
ent User Discussion <mailto:httpclient-users@hc.apache.org>=0ASent: Thu, Ma=
rch 3, 2011 1:06:41 PM=0ASubject: Re: Slowness of 4.1=0A=0AThanks Oleg for =
your explanation about comparitive vs real life. Most of the =0Atesting I d=
o is for "real life" since the end goal is user experience thus my =0Acomme=
nts. I can understand what Apache goals are better now and adjust my =0Aexp=
ectations accordingly.=0A=0AI did find what I needed as far as examples and=
 hope to get back to response =0Atime numbers I have seen with other client=
 side libraries and configurations.=0A=0AI will look at the links you provi=
ded to better understand what Apache hopes to =0Ado.=0A=0AThe only recommen=
dation for stats I would make is have a time measurement for =0Ahow long ea=
ch thread takes to execute a request. And show three new stats. One =0Afor =
average time based on that measurement, a 90% of measurements measurement =
=0A(Gives indication of distribution) and average response time over load f=
or each =0Aaverage. Just some thoughts you and the team can decide how if a=
t all those =0Asuggestions might fit your testing goals.=0A=0ARegards,=0A-T=
ony=0A=0A=0A=0A----- Original Message ----=0AFrom: Oleg Kalnichevski <olegk=
@apache.org>=0ATo: HttpClient User Discussion <mailto:httpclient-users@hc.a=
pache.org>=0ASent: Thu, March 3, 2011 1:52:37 PM=0ASubject: Re: Slowness of=
 4.1=0A=0AOn Thu, 2011-03-03 at 09:20 -0800, Tony Anecito wrote:=0A> Hi,=0A=
> =0A> Where can I find the client side code for this test? I think I found=
 it once =0A> before and discovered the way the response times were calcula=
ted were =0A>incorrect.=0A> What I saw indicated your test results were div=
ideing the number of requests =0A> into the time it took for all of them to=
 complete when the tests were run in =0A> parallel so it made it look like =
the requests were faster than they really =0A>were. =0A>=0A> It was like ta=
king ten 100ms requests run in parallel and getting 10ms per =0A> request w=
hen in reality it was 100ms.=0A> =0A=0AI did see your remarks about the ben=
chmark on the tomcat user list.=0A=0A(1) the benchmark tries to closely sim=
ulate Apache Bench (ab), which is=0Aa well established and widely used tool=
 for performance measurement.=0A=0Ahttp://en.wikipedia.org/wiki/ApacheBench=
=0A=0A(2) the benchmark is not intended to calculate 'real-life' or 'accura=
te'=0Aresponse time, whatever that means. The purpose of the benchmark is t=
o=0Aprovide a _baseline_ for a comparative performance analysis and is=0Ain=
tended to give a _rough_ indication as to whether an HTTP client A=0Afaster=
 than an client HTTP, or whether data throughput increased or=0Adecreased a=
fter particular set of changes.=0A=0AThat is it.=0A=0AHaving said all that =
you are very welcome to suggest improvements to the=0Abenchmark or a more a=
ccurate algorithm for calculating performance=0Anumbers.=0A=0AOleg =0A=0A> =
In either case I will see what if I can find again the client side code =0A=
> soemwhere and compare it to what I have unless you have a link to it.=0A>=
 =0A> Thanks,=0A> -Tony=0A> =0A> =0A> =0A> ----- Original Message ----=0A> =
From: Oleg Kalnichevski <olegk@apache.org>=0A> To: HttpClient User Discussi=
on <mailto:httpclient-users@hc.apache.org>=0A> Sent: Thu, March 3, 2011 2:4=
6:03 AM=0A> Subject: Re: Slowness of 4.1=0A> =0A> On Thu, 2011-03-03 at 01:=
38 -0800, Tony Anecito wrote:=0A> > Hi All,=0A> > =0A> > I downloaded httpc=
lient 4.1 and noticed it is significantly slower than 3.1. =0A=0A=0A>I =0A>=
=0A> =0A> > even used the threadsafe connection manager hoping for better p=
erformance. I =0A=0A=0A=0A> > used to get below 3msec and now it is above 1=
50msec.=0A> > Is keepalive used by default? If not where can I find an exam=
ple that sets it =0A>=0A>=0A>=0A> > using either a scheme, connection manag=
er httpclient instance or the httpget =0A=0A=0A=0A> > object?=0A> > =0A> > =
What kind of response time should I expect hitting a servlet with no code i=
n =0A=0A=0A=0A> >the =0A> >=0A> > method on a 2.8Ghz 6 core AMD server?=0A>=
 > =0A> > Any ideas on what it might be? I used code sample from the site a=
nd it works =0A=0A=0A=0A> > just very slow.=0A> > =0A> > Thanks,=0A> > -Ton=
y=0A> > =0A> =0A> HttpClient 4.1 is known to be comfortably faster than 3.1=
. There is=0A> likely to be a problem with how you are measuring performanc=
e / response=0A> time.=0A> =0A> http://wiki.apache.org/HttpComponents/HttpC=
lient3vsHttpClient4vsHttpCore =0A> =0A> Oleg=0A> =0A> =0A> =0A> -----------=
----------------------------------------------------------=0A> To unsubscri=
be, e-mail: =0A>mailto:mailto:mailto:mailto:httpclient-users-unsubscribe@hc=
.apache.org=0A> For additional commands, e-mail: =0A>mailto:mailto:mailto:m=
ailto:httpclient-users-help@hc.apache.org=0A> =0A> =0A>=A0 =A0 =A0 =0A> =0A=
> ---------------------------------------------------------------------=0A>=
 To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org=0A> For=
 additional commands, e-mail: httpclient-users-help@hc.apache.org=0A> =0A=
=0A=0A=0A------------------------------------------------------------------=
---=0ATo unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org=0A=
For additional commands, e-mail: httpclient-users-help@hc.apache.org=0A=0A=
=0A=0A=0A------------------------------------------------------------------=
---=0ATo unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org=0A=
For additional commands, e-mail: httpclient-users-help@hc.apache.org=0A=0A=
=0A      

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org