tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tony Anecito <adanec...@yahoo.com>
Subject ApacheHttp and Tomcat 7 tests...
Date Sat, 05 Mar 2011 08:00:43 GMT
Hi Oleg,

Summary The following test results show me that using the localhost as the 
server name and the ApacheHttp client running on the server as done with the 
ApacheHttp tests by Apache components group gets close to or bettter than 
expected. Using localhost I am guessing bypasses using the network and running 
500,000 requests over 20 threads gets a low average.

I still need to have the servlet setup to send data but I am wondering if the 
Apache component tests used Tomcat at all and instand just read a file from 
Apache Web Server. I will look further into this also to make sure I get closer 
to what the tests did.

Since I think I am getting closer to how Tomcat responds for a bare servlet with 
no code in it and maybe no network component I am more comfortable in 
understanding what Tomcat can truely do from a performance perspective.


If I go back to my 6 core server and run the same test (client running on the 
server where Tomcat is as the I did previously on my old laptop I get:

httpClient execute time in nanoseconds: 209391137
httpClient execute time in nanoseconds: 1595734
httpClient execute time in nanoseconds: 1298489
httpClient execute time in nanoseconds: 1238355
httpClient execute time in nanoseconds: 1262800
httpClient execute time in nanoseconds: 1196312
httpClient execute time in nanoseconds: 1253512
httpClient execute time in nanoseconds: 1457866
httpClient execute time in nanoseconds: 1215867
httpClient execute time in nanoseconds: 1254000
 
Concurrency level: 1
Time taken for tests: 0.234 seconds
Complete requests: 10
Failed requests: 0
Content transferred: 0 bytes
Requests per second: 42.735043 [#/sec] (mean)
Time per request: 23.4 [ms] (mean)

If I run next with the server name set to localhost instead of a internet url as 
done on the Apache tests I get (This should cause the network to be bypassed 
perhaps even the TCP/IP Stack? note the much lower response times):

httpClient execute time in nanoseconds: 100221746
httpClient execute time in nanoseconds: 890756
httpClient execute time in nanoseconds: 688845
httpClient execute time in nanoseconds: 688844
httpClient execute time in nanoseconds: 689823
httpClient execute time in nanoseconds: 614044
httpClient execute time in nanoseconds: 585200
httpClient execute time in nanoseconds: 604266
httpClient execute time in nanoseconds: 657066
httpClient execute time in nanoseconds: 408223
 
Concurrency level: 1
Time taken for tests: 0.109 seconds
Complete requests: 10
Failed requests: 0
Content transferred: 0 bytes
Requests per second: 91.74312 [#/sec] (mean)
Time per request: 10.9 [ms] (mean)


If I now run a 500,000 request thread over 20 sockets (localhost and running the 
client app on the Tomcat serverI get:

Concurrency level: 20
Time taken for tests: 14.549 seconds
Complete requests: 500000
Failed requests: 0
Content transferred: 0 bytes
Requests per second: 34366.625 [#/sec] (mean)
Time per request: 0.029098 [ms] (mean)



----- Original Message ----
From: Tony Anecito <adanecito@yahoo.com>
To: HttpClient User Discussion <httpclient-users@hc.apache.org>
Sent: Thu, March 3, 2011 10:49:53 PM
Subject: Re: Slowness of 4.1

Hi Oleg,

Here is some numbers when running TestHttpClient4. I am running from Eclipse 
using jdk 1. 6.0.22 and a Aspire 5670 which has Intel Core Duo T2300 (1.66 GHz, 
667 MHz FSB, 2MB L2 cache) and 2GB of DDR2 memory. Note how for a single thread 
running 10 requests the response time starts out very slow at 150msec then gets 
to 1.43msec at the end.

httpClient execute time in nanoseconds: 150181377
httpClient execute time in nanoseconds: 2180445
httpClient execute time in nanoseconds: 1540419
httpClient execute time in nanoseconds: 1757207
httpClient execute time in nanoseconds: 1759721
httpClient execute time in nanoseconds: 1492368
httpClient execute time in nanoseconds: 1493765
httpClient execute time in nanoseconds: 1474489
httpClient execute time in nanoseconds: 1456610
httpClient execute time in nanoseconds: 1437613
Document URI:  xxx
Document Length: 0 bytes
Concurrency level: 1
Time taken for tests: 0.187 seconds
Complete requests: 10
Failed requests: 0
Content transferred: 0 bytes
Requests per second: 53.475933 [#/sec] (mean)
Time per request: 18.7 [ms] (mean)


When I increase the number of threads to 50 and requests to 500000 I get a max 
req of 2.2K per second and around 400microseconds.
The CPU utilization is between 50-60% on the laptop and around 5-7% on my 6 core 

server with Tomcat 7. Tommow I will setup the test to run from a faster client 
and see the results.



----- Original Message ----
From: Tony Anecito <adanecito@yahoo.com>
To: HttpClient User Discussion <mailto:httpclient-users@hc.apache.org>
Sent: Thu, March 3, 2011 1:06:41 PM
Subject: Re: Slowness of 4.1

Thanks Oleg for your explanation about comparitive vs real life. Most of the 
testing I do is for "real life" since the end goal is user experience thus my 
comments. I can understand what Apache goals are better now and adjust my 
expectations accordingly.

I did find what I needed as far as examples and hope to get back to response 
time numbers I have seen with other client side libraries and configurations.

I will look at the links you provided to better understand what Apache hopes to 
do.

The only recommendation for stats I would make is have a time measurement for 
how long each thread takes to execute a request. And show three new stats. One 
for average time based on that measurement, a 90% of measurements measurement 
(Gives indication of distribution) and average response time over load for each 
average. Just some thoughts you and the team can decide how if at all those 
suggestions might fit your testing goals.

Regards,
-Tony



----- Original Message ----
From: Oleg Kalnichevski <olegk@apache.org>
To: HttpClient User Discussion <mailto:httpclient-users@hc.apache.org>
Sent: Thu, March 3, 2011 1:52:37 PM
Subject: Re: Slowness of 4.1

On Thu, 2011-03-03 at 09:20 -0800, Tony Anecito wrote:
> Hi,
> 
> Where can I find the client side code for this test? I think I found it once 
> before and discovered the way the response times were calculated were 
>incorrect.
> What I saw indicated your test results were divideing the number of requests 
> into the time it took for all of them to complete when the tests were run in 
> parallel so it made it look like the requests were faster than they really 
>were. 
>
> It was like taking ten 100ms requests run in parallel and getting 10ms per 
> request when in reality it was 100ms.
> 

I did see your remarks about the benchmark on the tomcat user list.

(1) the benchmark tries to closely simulate Apache Bench (ab), which is
a well established and widely used tool for performance measurement.

http://en.wikipedia.org/wiki/ApacheBench

(2) the benchmark is not intended to calculate 'real-life' or 'accurate'
response time, whatever that means. The purpose of the benchmark is to
provide a _baseline_ for a comparative performance analysis and is
intended to give a _rough_ indication as to whether an HTTP client A
faster than an client HTTP, or whether data throughput increased or
decreased after particular set of changes.

That is it.

Having said all that you are very welcome to suggest improvements to the
benchmark or a more accurate algorithm for calculating performance
numbers.

Oleg 

> In either case I will see what if I can find again the client side code 
> soemwhere and compare it to what I have unless you have a link to it.
> 
> Thanks,
> -Tony
> 
> 
> 
> ----- Original Message ----
> From: Oleg Kalnichevski <olegk@apache.org>
> To: HttpClient User Discussion <mailto:httpclient-users@hc.apache.org>
> Sent: Thu, March 3, 2011 2:46:03 AM
> Subject: Re: Slowness of 4.1
> 
> On Thu, 2011-03-03 at 01:38 -0800, Tony Anecito wrote:
> > Hi All,
> > 
> > I downloaded httpclient 4.1 and noticed it is significantly slower than 3.1. 


>I 
>
> 
> > even used the threadsafe connection manager hoping for better performance. I 



> > used to get below 3msec and now it is above 150msec.
> > Is keepalive used by default? If not where can I find an example that sets it 
>
>
>
> > using either a scheme, connection manager httpclient instance or the httpget 



> > object?
> > 
> > What kind of response time should I expect hitting a servlet with no code in 



> >the 
> >
> > method on a 2.8Ghz 6 core AMD server?
> > 
> > Any ideas on what it might be? I used code sample from the site and it works 



> > just very slow.
> > 
> > Thanks,
> > -Tony
> > 
> 
> HttpClient 4.1 is known to be comfortably faster than 3.1. There is
> likely to be a problem with how you are measuring performance / response
> time.
> 
> http://wiki.apache.org/HttpComponents/HttpClient3vsHttpClient4vsHttpCore 
> 
> Oleg
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: 
>mailto:mailto:mailto:mailto:httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: 
>mailto:mailto:mailto:mailto:httpclient-users-help@hc.apache.org
> 
> 
>      
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
> 



---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org




---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org


      

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message