hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Kalnichevski <ol...@apache.org>
Subject Re: Caching stateful HTTP clients in a pool
Date Mon, 26 Jan 2015 09:18:53 GMT
On Sun, 2015-01-25 at 22:24 +0100, Michael Osipov wrote:
> Am 2015-01-25 um 18:12 schrieb Oleg Kalnichevski:
> > On Sat, 2015-01-24 at 16:50 +0100, Michael Osipov wrote:
> >> Am 2015-01-24 um 12:29 schrieb Oleg Kalnichevski:
> >>> On Fri, 2015-01-23 at 23:36 +0100, Michael Osipov wrote:
> >>>> Hi folks,
> >>>>
> >>>> I am looking for a design decision to cache session information for
a
> >>>> request workflow embedded in a SOAP/REST-based webapp.
> >>>>
> >>>> Here is a little background how the workflow is structured and what
I am
> >>>> trying to solve:
> >>>>
> >>>> Note: I have read the entire tutorial and am somewhat overwhelmed by
the
> >>>> amount of features the HttpClient has to offer, so I beg your patience.
> >>>>
> >>>> Workflow:
> >>>> 1. Client issues a request either with SOAP or REST with an object id
> >>>> 2. Server accepts request and does the following:
> >>>>     2.1. Create a client with a cookie manager
> >>>>     2.2. Login to a backend server A (HTTP request, server A)
> >>>>     2.3. Read object information (HTTP request, server A)
> >>>>     2.4. Obtain a read ticket for a HTTP-based remote store (HTTP request,
> >>>> server A)
> >>>>     2.5. Download file behind object id to disk from remote store (HTTP
> >>>> request, server B)
> >>>>     2.6. perform a logout (HTTP request, server A)
> >>>>     2.7. Close the client
> >>>> 3. Respond to client with object information and the stream from the
> >>>> saved file
> >>>>
> >>>> The entire communication with the server A is stateful with a session
> >>>> cookie, all responses are fully consumed either with a response handler
> >>>> or with a try-with-resources clause.
> >>>>
> >>>> The problem: the login/logout requests consume around 10 seconds,
> >>>> sometimes even more. I'd like to avoid this every time.
> >>>>
> >>>> My idea was to come around this with Commons Pool 2 which I have already
> >>>> used for some other session pooling successfully.
> >>>>
> >>>> Requirements: it is not only necessary to maintain the cookie store
but
> >>>> a client id header which has a randomly generated integer assigned plus
> >>>> a request counter. The maintained session has to be used by one thread
> >>>> at the same time, same applies for the header otherwise server A would
> >>>> queue all requests. The pool would guarantee that.
> >>>>
> >>>> Two possible solutions come to my mind:
> >>>>
> >>>> 1. Save the client id and cookie store in the pool and always create
a
> >>>> new client.
> >>>> 2. Save the client id along with the authenticated HttpClient and reuse
it.
> >>>>
> >>>> Both solutions will always increment the request counter of course.
> >>>>
> >>>>    From HttpClient's view which makes more sense here? What if I go
with
> >>>> option 2, will I have stale connections, clients, etc.?
> >>>>
> >>>> Thanks,
> >>>>
> >>>> Michael
> >>>>
> >>>
> >>> Hi Michael
> >>>
> >>> How about a slightly different approach? Consider using one HttpClient
> >>> for all sessions and maintaining a pool of HttpContext instances with a
> >>> cookie store / session id.
> >>
> >> Hi,
> >>
> >> thanks for this tip, you are proposing a very interesting approach which
> >> I have not considered due to missing knowledge.
> >>
> >> I would have to do the following/solve following questions:
> >>
> >> 1. Create a vanilla HttpClient in my beans.xml as a singleton. Please
> >> note that the web application can run for days if not even for weeks and
> >> the HttpClient would remain open. That is not a problem?
> >
> > No, it should not be. You should however consider employing a connection
> > evictor thread as described here:
> >
> > http://hc.apache.org/httpcomponents-client-4.3.x/tutorial/html/connmgmt.html#d5e405
> 
> If I won't I might end up in stale connections right? This is at least 
> what the docs tell me.
> 

No matter what you do there is always a possibility of a connection
getting 'stale' immediately after passing a staleness check. A
combination of a reasonable eviction policy enforced by a background
thread and a reasonable recovery (request retrial) policy should be
sufficient to ensure near 100% reliability of request execution in my
opinion.

Oleg


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org


Mime
View raw message