cxf-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Kalnichevski <ol...@apache.org>
Subject Re: Async http client experiments....
Date Fri, 03 Aug 2012 09:29:41 GMT
On Thu, 2012-08-02 at 14:58 -0400, Daniel Kulp wrote:
> On Wednesday, August 01, 2012 05:58:23 PM Oleg Kalnichevski wrote:
> > Hi Daniel
...
>  
> > I also noticed that for some reason additional connections get open and
> > closed without transmitting any data, which obviously should not be
> > happening. I'll investigate the problem if you confirm that you want me
> > to proceed.
> 
> Interesting.  I'm actually not seeing that now.  :-)   I ran the testCalls 
> things with small and larger requests and in both cases, all 20K requests 
> went on a single connection.  Was quite impressed.  (I actually expected the 
> Jetty backend to have a limit on number of requests per keepalive as I ran 
> into a similar setting in Tomcat last week that defaulted to 100).
> 

I would like to investigate nonetheless. Besides, connection pool
logging might come handy in other situations as well.

> I just went through and added a little bit of error handling into it as 
> well.   Mostly around a connection refused exception and for read timeouts.   
> For the read timeout, I DID have to add an ioct.shutdown() in the 
> SharedInBuffer consume method when read or it kept getting called repeatedly 
> since we didn't read anything.
> 

Hmmm. This should not be necessary. It is the responsibility of the
protocol handler to shut down things properly if connection goes down or
something breaks at runtime. In case of an I/O timeout the protocol
handler always shuts down the execution handler, which in its turn
should shut down shared i/o buffers [1]. I'll look into it.


> Also started working on the resends and such, but haven't yet added a 
> testcase for that.  
> 
> So, stuff I have left to look at are basically:  (and I've add //FIXME 
> comments in the code where I *think* something needs to happen)
> 
> 1) Proxy support
> 2) SSL/TLS
> 3) Connection timeouts
> 

Connect timeouts, i/o timeouts, TLS/SSL layering should be supported out
of the box. Proxy support can present a certain conceptual challenge as
it automatically implies a decision as to whether or not HTTP
authentication is to be supported and if so to what extent. Complex auth
schemes such SPNEGO and NTLM entail a lot of additional complexity and
may require additional external dependencies. HC provide full support
for HTTP request proxying and HTTPS tunneling in its higher level
components (HttpClient and HttpAsyncClient) but not in core. So, the
options are to upgrade to HttpAsyncClient or to implement proxy support
in the CXF HTTP conduit. The former most likely means having to live
with different (potentially inconsistent) HTTP authentication
frameworks. The latter means having more flexibility at the price of
having more code to maintain. So, essentially it is a classic buy
(re-use) or build dilemma.

> CXF allows these to be configured on a per-client (and sometimes even per-
> request) basis.    It looks like HC seems to have these on the connection 
> factory.

It does not have to be this way. Connections can start off as plain and
later get upgraded to TLS/SSL by the protocol handling code. One would
have to tweak the default protocol handler a bit, though, probably by
overriding the HttpAsyncRequestExecutor#requestReady method [2]. 

>   I thought about a new factory per request, but the pool seems to 
> be tied to the connection factory  as well and a full pool per request would 
> be aweful.   I then wanted to see if I could pass some sort of contextual 
> object in (there is that BasicHttpContext object) that I could pull 
> information out of at connection time or similar, but I could really figure 
> out where that went, at least not until after the IOSession is setup, which 
> is too late.   Also thought about a thread local, but that doesn't work with 
> the async connection.   
> 

I'll try to find time to write all the necessary plumbing code on this
weekend.

Cheers

Oleg

> Any ideas on that would be a huge help.  I'll keep digging a bit as well.   
> Kind of learning more about the HC stuff and NIO and all kind of things, 
> which is kind of fun.   :-)
> 
> 

[1]
http://hc.apache.org/httpcomponents-core-ga/httpcore-nio/xref/org/apache/http/nio/protocol/HttpAsyncRequestExecutor.html#269
[2]
http://hc.apache.org/httpcomponents-core-ga/httpcore-nio/xref/org/apache/http/nio/protocol/HttpAsyncRequestExecutor.html#124



Mime
View raw message