hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Russell Brown <russel...@basho.com>
Subject Re: Using ThreadSafeClientConnManager
Date Thu, 04 Aug 2011 16:17:24 GMT

On 4 Aug 2011, at 17:07, Sam Crawford wrote:

> It depends on what you want. What I said in my post was that if you
> wanted at most 100 connections per VIP, then you should use that
> config. If you don't want to artificially limit them at all and you
> just want to go flat out between them, then I agree you should use
> MaxPerRoute == MaxTotal. And you'd probably want the number a lot
> higher than 100 !

For sure. It didn't notice that you introduced the caveat of so many connections per host
for the OP. My mistake. So yeah, what Sam said :D

> 
> Thanks,
> 
> Sam
> 
> 
> On 4 August 2011 17:01, Russell Brown <russelldb@basho.com> wrote:
>> 
>> On 4 Aug 2011, at 16:56, Sam Crawford wrote:
>> 
>>> No, that would be a bad idea. MaxTotal should be >= MaxPerRoute. In
>>> your case it should be at least double it.
>>> 
>>> If you have two VIPs and wish to evenly balance load between them
>>> (with at most 100 concurrent connections per VIP), then you'd use:
>>> 
>>> cm.setMaxTotal(200);
>>> cm.setDefaultMaxPerRoute(100);
>> 
>> Really??? That sounds terrible. If on vip was tickling under at 10 connections, but
the other was at 100 then you would have blocking *and* capacity to spare.
>> 
>> I need to look at the implementation (I haven't read the source in a while) but I
would hope the max per route should be == to max total.
>> 
>> 
>>> 
>>> Thanks,
>>> 
>>> Sam
>>> 
>>> 
>>> 
>>> On 4 August 2011 16:50, Mohit Anchlia <mohitanchlia@gmail.com> wrote:
>>>> On Thu, Aug 4, 2011 at 8:29 AM, Sam Crawford <samcrawford@gmail.com>
wrote:
>>>>> No. What you've done with cm.setMaxTotal(100) is set 100 maximum
>>>>> connections across the entire connection manager. So, with that alone,
>>>>> you could only reach 100 concurrent connections if you were hitting at
>>>>> least 50 unique hosts.
>>>>> 
>>>>> If you want 100 connections to a single host and you were only making
>>>>> connections to that one host, then you could use something simple like
>>>>> this:
>>>>> 
>>>>> cm.setMaxTotal(100);
>>>>> cm.setDefaultMaxPerRoute(100);
>>>> 
>>>> Thanks! so if I have 2 unique VIP that I use in HttpGet/Post etc. for
>>>> shared HttpClient then I use cm.setMaxTotal(2) and
>>>> cm.setDefaultMaxPerRoute(100);
>>>>> 
>>>>> Thanks,
>>>>> 
>>>>> Sam
>>>>> 
>>>>> On 4 August 2011 16:18, Mohit Anchlia <mohitanchlia@gmail.com>
wrote:
>>>>>> On Thu, Aug 4, 2011 at 7:05 AM, Sam Crawford <samcrawford@gmail.com>
wrote:
>>>>>>> Forgive the poor wording - I meant to say "The HTTP/1.1 RFC says
you
>>>>>>> should make _at most_ 2 connections per target host".
>>>>>>> 
>>>>>> Sorry I am little confused. In my example I thought I was creating
100
>>>>>> connections to a particular host by setting setMaxTotal to 100.
>>>>>>> 
>>>>>>> On 4 August 2011 15:03, Sam Crawford <samcrawford@gmail.com>
wrote:
>>>>>>>> Yes, absolutely. The HTTP/1.1 RFC says you should make 2
connections
>>>>>>>> per host, and HttpClient obeys that strictly out-of-the-box
(side
>>>>>>>> note: web browsers do not anymore).
>>>>>>>> 
>>>>>>>> A 'route' in HttpClient is typically defined as protocol
(HTTP/HTTPS)
>>>>>>>> + host + port. I see there are some additional options in
4.1.x for
>>>>>>>> tunneling, but the three I listed will be the primary ones.
>>>>>>>> 
>>>>>>>> See section 2.8.4 in
>>>>>>>> http://hc.apache.org/httpcomponents-client-ga/tutorial/html/connmgmt.html.
>>>>>>>> You'll want to change either the default max per route or
the max for
>>>>>>>> an individual route.
>>>>>>>> 
>>>>>>>> Hope this helps,
>>>>>>>> 
>>>>>>>> Sam
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On 4 August 2011 14:55, KARR, DAVID (ATTSI) <dk068x@att.com>
wrote:
>>>>>>>>>> -----Original Message-----
>>>>>>>>>> From: Russell Brown [mailto:russelldb@basho.com]
>>>>>>>>>> Sent: Thursday, August 04, 2011 12:04 AM
>>>>>>>>>> To: HttpClient User Discussion
>>>>>>>>>> Subject: Re: Using ThreadSafeClientConnManager
>>>>>>>>>> 
>>>>>>>>>> On 4 Aug 2011, at 01:18, Mohit Anchlia wrote:
>>>>>>>>>> 
>>>>>>>>>>> I am trying to understand these 2:
>>>>>>>>>>> 
>>>>>>>>>>> ClientConnectionManager cm = new ThreadSafeClientConnManager();
>>>>>>>>>>> cm.setMaxTotal(100);
>>>>>>>>>>> HttpClient httpClient = new DefaultHttpClient(cm);
>>>>>>>>>>> 
>>>>>>>>>>> Have couple of questions:
>>>>>>>>>>> 
>>>>>>>>>>> 1) Can one httpClient object be shared accross
multiple threads doing
>>>>>>>>>>> post/get/delete/put requests?
>>>>>>>>>> 
>>>>>>>>>> Yes.
>>>>>>>>>> 
>>>>>>>>>>> 2) What happens in above case if there are more
concurrent threads
>>>>>>>>>> say
>>>>>>>>>>> 200 than actual no. of connections? So what happens
if 200 threads
>>>>>>>>>>> trying to use httpClient object (shared). But
we created only 100
>>>>>>>>>>> connections. Does it wait on .execute if all
connections are busy?
>>>>>>>>>> 
>>>>>>>>>> The thread attempting to acquire a connection blocks.
If you have set a
>>>>>>>>>> timeout then it blocks until the timeout is reached
and an exception is
>>>>>>>>>> thrown.
>>>>>>>>>> 
>>>>>>>>>> Worth noting, in your above snippet the max connections
per route will
>>>>>>>>>> only be *2*.
>>>>>>>>> 
>>>>>>>>> Sorry to the OP to hijack this question, but I'd like
to understand a little more about this.  Is a "route" defined as the path to a particular
host?  If all of my connections go to the same host, and I'll have numerous threads trying
to make that connection, do I need to change some default to allow more than 2 connections
per "route"?
>>>>>>>>> 
>>>>>>>>>>> Can somone please help me understand?
>>>>>>>>>>> 
>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>>>>>>>>>>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>>>>>>>>>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>>>>>>>>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>>>>>>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>>>>>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>>>>>> 
>>>>>> 
>>>>> 
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>>>>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>>>>> 
>>>>> 
>>>> 
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>>>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>>>> 
>>>> 
>>> 
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>>> 
>> 
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>> 
>> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org


Mime
View raw message