cxf-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mandy Warren <mandys.in...@gmail.com>
Subject Re: Async client Http transport and keep-alive timeouts
Date Mon, 22 Dec 2014 21:11:47 GMT
Hi Dan 

At the moment our code is based on the 2.7.x branch, we hadn't planned to upgrade to 3 til
early next year. However maybe this will give me the incentive to upgrade sooner!

Can I just confirm that if we want to use a later version of httpcore we need to upgrade to
3.0.x? I seem to remember an earlier post where you said you couldn't upgrade 2.7.x due to
api incompatibilities?

Thanks
Mandy

Sent from a mobile device

> On 22 Dec 2014, at 18:39, Daniel Kulp <dkulp@apache.org> wrote:
> 
> 
> This is likely down in HTTP Client someplace, but I am questioning one thing:  if I use
wireshark to grab the raw messages, I’m seeing the Connection: Keep-Alive header:
> 
> POST /SoapContext/SoapPort HTTP/1.1
> Accept: */*
> User-Agent: Apache CXF ${project.version}
> SOAPAction: ""
> Content-Length: 615
> Content-Type: text/xml; charset=UTF-8
> Host: localhost:9001
> Connection: Keep-Alive
> 
> At least with the latest code on trunk, but that shouldn’t be any different than what’s
on 3.0.x.   2.7.x does use an older version of the HTTP components libraries so there might
be some issues there.   Could you try updating to 3.0.3 and seeing if the problems still remain?
> 
> 
> Dan
> 
> 
> 
>> On Dec 21, 2014, at 5:40 PM, Mandy Warren <mandys.inbox@gmail.com> wrote:
>> 
>> Hi Sergey, I tried using HTTPUrlConnection for my synchronous calls and it worked
very nicely, I did a tcpdump and could see it respected the keep alive setting and retried
with a new connection if a connection was unexpectedly closed by the server.
>> 
>> I would however like the option to also use async transport as we are looking at
using the async/Future methods on WebClient. I do feel there is a bug in it but before I raise
a jira it would be good to get Dan/Aki or Oleg's opinions.
>> 
>> I am also happy to send the tcpdump output for the async transport test showing the
FIN from the server and subsequent processing if this helps. Note I have also tried explicitly
setting the Connection: keep-alive header but still no luck..
>> 
>> Many thanks
>> Mandy
>> 
>> Sent from my iPad
>> 
>>> On 16 Dec 2014, at 12:36, Sergey Beryozkin <sberyozkin@gmail.com> wrote:
>>> 
>>> Hi Mandy
>>> 
>>> I'd like to ask Dan, Aki or Oleg to comment - hopefully one of HTTP Client experts
would read it :-)
>>> 
>>> by the way, what about HTTPUrlConnection - does it work less efficiently ?
>>> 
>>> Thanks, Sergey
>>>> On 12/12/14 14:23, Mandy Warren wrote:
>>>> Hi,
>>>> 
>>>> We are using cxf version 2.7.12 and are using Asynchronous Client HTTP Transport
to send a request to an external service. This external service runs on an Apache server and
is configured as follows:-
>>>> 
>>>> Keep-Alive: timeout=2, max=100
>>>> Connection: Keep-Alive
>>>> 
>>>> i.e. the external server will close the connection after 2 seconds of idle
time.
>>>> 
>>>> Under heavy load, every so often we see an attempt to send a request on a
connection fail because at the exact same time, the external server has closed the connection.
I would then expect CXF to retry the sending of the request again but it doesn't, the call
just fails. Here is some relevant debug to indicate the issue:
>>>> 
>>>> 2014-12-09 16:10:29,090 [I/O dispatcher 2] DEBUG org.apache.http.headers
[txId=|client=|entryService=|callingService=] [] [http-outgoing-268 >> POST /Service/ws/
Controls/1.0/GetStuff HTTP/1.1]
>>>> 2014-12-09 16:10:29,090 [I/O dispatcher 2] DEBUG org.apache.http.headers
[txId=|client=|entryService=|callingService=] [] [http-outgoing-268 >> txId: 1234]
>>>> 2014-12-09 16:10:29,090 [I/O dispatcher 2] DEBUG org.apache.http.headers
[txId=|client=|entryService=|callingService=] [] [http-outgoing-268 >> User-Agent: Apache
CXF 2.7.12]
>>>> 2014-12-09 16:10:29,090 [I/O dispatcher 2] DEBUG org.apache.http.headers
[txId=|client=|entryService=|callingService=] [] [http-outgoing-268 >> SOAPAction: executeService]
>>>> 2014-12-09 16:10:29,090 [I/O dispatcher 2] DEBUG org.apache.http.headers
[txId=|client=|entryService=|callingService=] [] [http-outgoing-268 >> Accept: */*]
>>>> 2014-12-09 16:10:29,090 [I/O dispatcher 2] DEBUG org.apache.http.headers
[txId=|client=|entryService=|callingService=] [] [http-outgoing-268 >> Content-Length:
1547]
>>>> 2014-12-09 16:10:29,090 [I/O dispatcher 2] DEBUG org.apache.http.headers
[txId=|client=|entryService=|callingService=] [] [http-outgoing-268 >> Content-Type:
application/xml]
>>>> 2014-12-09 16:10:29,090 [I/O dispatcher 2] DEBUG org.apache.http.headers
[txId=|client=|entryService=|callingService=] [] [http-outgoing-268 >> Host: a-b.x.com:80]
>>>> 2014-12-09 16:10:29,090 [I/O dispatcher 2] DEBUG org.apache.http.impl.nio.reactor.IOSessionImpl
[txId=|client=|entryService=|callingService=] [] [http-outgoing-268 xxx.x.xx.xx:xxxxx<->yyy.y.yyy.yy:yy[ACTIVE][rw:w]:
Event set [w]]
>>>> 2014-12-09 16:10:29,090 [I/O dispatcher 2] DEBUG org.apache.http.impl.nio.reactor.IOSessionImpl
[txId=|client=|entryService=|callingService=] [] [http-outgoing-268xxx.x.xx.xx:xxxxx<->yyy.y.yyy.yy:yy[ACTIVE][rw:rw]:
-1 bytes read]
>>>> 2014-12-09 16:10:29,091 [I/O dispatcher 2] DEBUG org.apache.cxf.transport.http.asyncclient.AsyncHTTPConduitFactory$4
[txId=|client=|entryService=|callingService=] [] [[exchange: 9268] aborting connection [id:268][route:{}->http://a-b.x.com:80][state:null]]
>>>> 2014-12-09 16:10:29,091 [I/O dispatcher 2] DEBUG org.apache.http.impl.nio.reactor.IOSessionImpl
[txId=|client=|entryService=|callingService=] [] [http-outgoing-268xxx.x.xx.xx:xxxxx<->yyy.y.yyy.yy:yy[ACTIVE][rw:rw]:
Shutdown]
>>>> 2014-12-09 16:10:29,091 [I/O dispatcher 2] DEBUG org.apache.http.impl.nio.reactor.IOSessionImpl
[txId=|client=|entryService=|callingService=] [] [http-outgoing-268 0.0.0.0:11111<->yyy.y.yyy.yy:yy[CLOSED][]:
Shutdown]
>>>> 2014-12-09 16:10:29,091 [I/O dispatcher 2] DEBUG org.apache.cxf.transport.http.asyncclient.AsyncHTTPConduitFactory$3
[txId=|client=|entryService=|callingService=] [] [Connection released: [id: 268][route: {}->http://a-b.x.com:80][total
kept alive: 0; route allocated: 1 of 1000; total allocated: 1 of 5000]
>>>> 
>>>> The only thing I've noticed which could be relevant is that when we send
a request to the external service, CXF doesn't explicitly send the Keep-alive headers (as
it is implicit through the use of HTTP 1.1) which means the external service does not send
back the explicit Keep-Alive: timeout=2, max=100 header.
>>>> 
>>>> Please could someone confirm whether or not you would expect the async transport
to retry sending the request when a connection is closed due to a keep-alive idle timeout?
Am I missing some configuration to allow this?
>>>> 
>>>> Many thanks
>>>> Mandy
>>>> 
>>>> 
>>>> 
>>>> Sent from a mobile device
> 
> -- 
> Daniel Kulp
> dkulp@apache.org - http://dankulp.com/blog
> Talend Community Coder - http://coders.talend.com
> 

Mime
View raw message