Mailing-List: contact httpclient-users-help@hc.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "HttpClient User Discussion" <httpclient-users@hc.apache.org>
Received-SPF: error (nike.apache.org: local policy)
Message-ID: <1373535323.8015.6.camel@ubuntu>
Subject: Re: Scalable Http Client
From: Oleg Kalnichevski <olegk@apache.org>
To: HttpClient User Discussion <httpclient-users@hc.apache.org>
Date: Thu, 11 Jul 2013 11:35:23 +0200
In-Reply-To: 
 <a5fd6e7abeb7441e86365ebce9bf3380@BY2PR05MB144.namprd05.prod.outlook.com>
References: 
	<2b7ca3a05cac474bafb16da2d82581ee@BY2PR05MB144.namprd05.prod.outlook.com>
	 <CAJqdOe-wiC7NN+B4D3QQtmeT7tDAzQBL0qg5_g-wy3gGEhG7FA@mail.gmail.com>
	 <a5fd6e7abeb7441e86365ebce9bf3380@BY2PR05MB144.namprd05.prod.outlook.com>
Content-Type: text/plain; charset="UTF-8"
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit

On Thu, 2013-07-11 at 09:24 +0000, Dave Johnston wrote:
> 24 logical cores - 1 server.
> I'm trying to simulate real world traffic, where the server I'm testing against handles 30K + TPS.  So I'm creating 1000 connections each doing between 10 or 30 requests each.
> 

In this case you should have a pool of HttpClient instances each
configured to use a small pool of connections (2 to 5). This setup will
be more representative of thousands of concurrent browser connections.
With one large pool of connections you basically have 1000 thousand of
threads constantly contending for one global pool lock.

Hope this helps

Oleg 

> I've been able to create up to 2000 connections each making 10 requests each per second with other load test tools (tsung), but I'm interested to see if I can  do that by using the apache client, as it means I can hook it into other tools we have for automation.
> 
> 
> -----Original Message-----
> From: Ja kub [mailto:jjakub83@gmail.com] 
> Sent: 11 July 2013 10:19
> To: HttpClient User Discussion
> Subject: Re: Scalable Http Client
> 
> Dave,
> 
> Does this mean 1000 concurrent threads ?
> How many processors do you have ? How many servers ?
> I suppose, that with so many threads latency of every operation increases, try to get latency of 1000xSystem.out.println instead of httpClient.execute(httpget, context); and send results comparison here.
> Please send how much latency increases now ?
> So or so, very interesting case.
> 
> BR
> Jakub
> 
> 
> On Wed, Jul 10, 2013 at 3:49 PM, Dave Johnston <Dave.Johnston@owmobility.com
> > wrote:
> 
> > Hi,
> > I've been trying to execute a large volume of transactions to simulate 
> > real traffic using the httpclient.  I'm trying to create 1000 
> > concurrent connections, that each perform 10 requests per second.  
> > i.e. 10000 transactions per second.
> >
> > I've based my code on the multi-threaded example:
> >
> > http://hc.apache.org/httpcomponents-client-ga/httpclient/examples/org/
> > apache/http/examples/client/ClientMultiThreadedExecution.java
> >
> > I've increased the number of connections allowed per route to 1000 and 
> > have attempted to measure latency by doing this:
> >
> > long currentTime = System.currentTimeMillis(); HttpResponse response = 
> > httpClient.execute(httpget, context); long executionTime = 
> > System.currentTimeMillis();
> >
> > System.out.printf("ID [%d] Executed in %d milliseconds\n", id, 
> > (executionTime-currentTime));
> >
> > It looks like the latency increases as the number of connections go 
> > up, although I would have hoped to see it stay the same (I've used 
> > other load test tools to execute 20K transactions with steady latency).
> >
> > Whats the best approach for writing an application that behaves this way?
> >  Is the standard PoolingClientConnectionManager with multiple threads 
> > the way to go?
> >
> >
> >
> >
> >
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org