cxf-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Kulp <dk...@apache.org>
Subject Re: Async http client experiments....
Date Tue, 04 Sep 2012 19:03:37 GMT

Finally had a chance over the last week or so to re-dig into the async http stuff.   You've
probably seen a lot of commits and such related to it.  :-)

Good news:
ALL of the CXF system tests now pass using it instead of the URLConnection based transport.
 I'm pretty excited about that as that's a huge milestone.  

That said, there is still a bit of work todo:

1) Proxy support - haven't looked into this at all yet.   Hard for me to test as well as I
don't have any sort of proxy setup anywhere.  I may need to look into how to setup a simple
proxy to dig into this.

2) HTTPs Keep-Alive - right now, with HTTPs, a new connect is created for every request. 
 With HTTPs, that's really expensive as it needs to do the handshakes and such for every request.
    This is next on my TODO to look at.   One issue is that the BasicNIOConnPool  really only
compares the host name/port/scheme to determine connection re-use.  Not any of the certs or
anything like that.   I'll need to implement a "state" object to handle checking a lot of
that.   Definitely next.  

3) Configuration - now that things are working, I'll need to find a way to expose some configuration
things.  Specifically around things like number of connections to hold open, number per target/host,
thread pools, etc….

4) Flip the logging and such to using our normal Logging framework.

5) Move it all from the sandbox to trunk.  


Anyway, definitely getting closer.   :-)


Dan



On Aug 6, 2012, at 10:43 AM, Oleg Kalnichevski <olegk@apache.org> wrote:

> On Fri, 2012-08-03 at 09:17 -0400, Daniel Kulp wrote:
>> On Friday, August 03, 2012 11:29:41 AM Oleg Kalnichevski wrote:
>>>> I just went through and added a little bit of error handling into it as
>>>> well.   Mostly around a connection refused exception and for read
>>>> timeouts. For the read timeout, I DID have to add an ioct.shutdown() in
>>>> the SharedInBuffer consume method when read or it kept getting called
>>>> repeatedly since we didn't read anything.
>>> 
>>> Hmmm. This should not be necessary. It is the responsibility of the
>>> protocol handler to shut down things properly if connection goes down or
>>> something breaks at runtime. In case of an I/O timeout the protocol
>>> handler always shuts down the execution handler, which in its turn
>>> should shut down shared i/o buffers [1]. I'll look into it.
>> 
>> Well, it's mostly because I didn't actually set the timeouts on the 
>> connection at all.   Haven't figured that out yet mostly due to the same 
>> issus as with the SSL/connection timeouts/etc...  as we set them on a per-
>> request basis normally.   For now so things don't "hang", I just added 
>> timeouts on the wait(...) calls we use when we need to block for additional 
>> IO.    In that case, the wait(...) times out and I need to shutdown the 
>> buffers and such manually so when the data DOES eventually arrive, we ignore 
>> it.   Setting the "real" timeouts on the connections would definitely help.
>> 
> 
> 
> Hi Daniel
> 
> I tweaked the code a bit to make sure i/o timeouts (both connect and
> receive ones) are handled correctly and can be specified on a per
> request basis overriding the defaults. Please review / verify.
> 
> I also added connection pool lease / release logging. I am no longer
> seeing anything suspicious related to connection management. When
> executing requests sequentially the connection manager allocates one
> connection only. 
> 
>>>> CXF allows these to be configured on a per-client (and sometimes even
>>>> per- request) basis.    It looks like HC seems to have these on the
>>>> connection factory.
>>> 
>>> It does not have to be this way. Connections can start off as plain and
>>> later get upgraded to TLS/SSL by the protocol handling code. One would
>>> have to tweak the default protocol handler a bit, though, probably by
>>> overriding the HttpAsyncRequestExecutor#requestReady method [2].
>> 
>> Ah.  OK.   I'll take a look there.   I was looking in the various connection 
>> factories and pools.   Haven't looked there yet.   Lots of stuff going on.  
>> :-)
>> 
> 
> I also added a CXF specific connection manager implementation in order
> to enable SSL customization based on request configuration. However,
> before I proceed I would like to understand the requirements a little
> better. Can I assume that once an SSL connection has been fully set up
> using request level config parameters and can be pooled and safely
> reused for subsequent requests to the same service (even by different
> users) or requests may contain different / incompatible SSL
> configuration for the same service?   
> 
> Oleg   
> 
> 

-- 
Daniel Kulp
dkulp@apache.org - http://dankulp.com/blog
Talend Community Coder - http://coders.talend.com


Mime
View raw message