hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Osipov <micha...@apache.org>
Subject Re: Caching stateful HTTP clients in a pool
Date Sun, 25 Jan 2015 21:24:01 GMT
Am 2015-01-25 um 18:12 schrieb Oleg Kalnichevski:
> On Sat, 2015-01-24 at 16:50 +0100, Michael Osipov wrote:
>> Am 2015-01-24 um 12:29 schrieb Oleg Kalnichevski:
>>> On Fri, 2015-01-23 at 23:36 +0100, Michael Osipov wrote:
>>>> Hi folks,
>>>> I am looking for a design decision to cache session information for a
>>>> request workflow embedded in a SOAP/REST-based webapp.
>>>> Here is a little background how the workflow is structured and what I am
>>>> trying to solve:
>>>> Note: I have read the entire tutorial and am somewhat overwhelmed by the
>>>> amount of features the HttpClient has to offer, so I beg your patience.
>>>> Workflow:
>>>> 1. Client issues a request either with SOAP or REST with an object id
>>>> 2. Server accepts request and does the following:
>>>>     2.1. Create a client with a cookie manager
>>>>     2.2. Login to a backend server A (HTTP request, server A)
>>>>     2.3. Read object information (HTTP request, server A)
>>>>     2.4. Obtain a read ticket for a HTTP-based remote store (HTTP request,
>>>> server A)
>>>>     2.5. Download file behind object id to disk from remote store (HTTP
>>>> request, server B)
>>>>     2.6. perform a logout (HTTP request, server A)
>>>>     2.7. Close the client
>>>> 3. Respond to client with object information and the stream from the
>>>> saved file
>>>> The entire communication with the server A is stateful with a session
>>>> cookie, all responses are fully consumed either with a response handler
>>>> or with a try-with-resources clause.
>>>> The problem: the login/logout requests consume around 10 seconds,
>>>> sometimes even more. I'd like to avoid this every time.
>>>> My idea was to come around this with Commons Pool 2 which I have already
>>>> used for some other session pooling successfully.
>>>> Requirements: it is not only necessary to maintain the cookie store but
>>>> a client id header which has a randomly generated integer assigned plus
>>>> a request counter. The maintained session has to be used by one thread
>>>> at the same time, same applies for the header otherwise server A would
>>>> queue all requests. The pool would guarantee that.
>>>> Two possible solutions come to my mind:
>>>> 1. Save the client id and cookie store in the pool and always create a
>>>> new client.
>>>> 2. Save the client id along with the authenticated HttpClient and reuse it.
>>>> Both solutions will always increment the request counter of course.
>>>>    From HttpClient's view which makes more sense here? What if I go with
>>>> option 2, will I have stale connections, clients, etc.?
>>>> Thanks,
>>>> Michael
>>> Hi Michael
>>> How about a slightly different approach? Consider using one HttpClient
>>> for all sessions and maintaining a pool of HttpContext instances with a
>>> cookie store / session id.
>> Hi,
>> thanks for this tip, you are proposing a very interesting approach which
>> I have not considered due to missing knowledge.
>> I would have to do the following/solve following questions:
>> 1. Create a vanilla HttpClient in my beans.xml as a singleton. Please
>> note that the web application can run for days if not even for weeks and
>> the HttpClient would remain open. That is not a problem?
> No, it should not be. You should however consider employing a connection
> evictor thread as described here:
> http://hc.apache.org/httpcomponents-client-4.3.x/tutorial/html/connmgmt.html#d5e405

If I won't I might end up in stale connections right? This is at least 
what the docs tell me.

>> 2. To make the client above usable concurrently, would I need to create
>> a connection pool? I do not intend to use long-lived connections to
>> server A.
> You do not have to create a connection pool. HttpClient uses a pool of
> connections by default.

Yes, but it is limited to two concurrent connections by default. I have 
to tweak it anyany.

>> 3. Before login is called, I assign a cookie store to the HttpContext [1]?
>> So I would still need to carry on the cookie store plus increment the
>> request counter and supply the client id to the HTTP request?
> You just need to create a separate CookieStore per HttpContext.
>> I am having trouble to understand the difference between HttpContext and
>> HttpClientContext.
> HttpContext is effectively just a hash map. HttpClientContext is merely
> convenience class that makes that map behave more like a POJO.

By using it, you are probably referring to [1].

Thanks for the advisory, highly appreciated. I think, I have now the 
insight how this has to/can be coded. I'll try this in February and will 
respond how the outcome will look like.



To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org

View raw message