cxf-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sergey Beryozkin <sberyoz...@gmail.com>
Subject Re: Dealing with CLOSE_WAIT connections
Date Thu, 06 Aug 2015 09:23:16 GMT
Hi

It can be enabled on a per proxy or webclient instance,

WebClient.getConfig(proxy).getResponseContext().put("response.stream.auto.close", 
true);

Indeed, as far the client runtime is concerned, it can auto-close by 
default but can avoid it for some well-known types (ex, Document, 
Source), but in our tooling we have the code reading DOM4J, and passing 
it to other components, and checking DOM4J in the CXF client module 
becomes problematic, etc, it is rather brittle. It is indeed safer IMHO 
to selectively enable the auto-closure or manually close it...

It can be debated if users having the code consuming Document(DOM or 
DOM4J) /Source/ etc should themselves disable the auto-closure with the 
runtime enabling it by default. I'm not sure, by default the runtime 
should work for all the cases...

Cheers, Sergey
On 06/08/15 00:55, Xiaobin wrote:
> Appreciate the clarification, Sergey!
>
> One more further question, while using proxy based client, is it
> possible/good-practice to conditionally enable/disable auto-closure of
> stream ?     For example: we generate a proxy based on class Foo,
> Foo.getA() needs to keep the stream open, while Foo.getB(), Foo.getC() and
> a lot more other endpoints wants it to auto close the stream. Is there a
> way to achieve that ?
>
> Thanks,
> Xiaobin
>
> 2015-08-05 15:54 GMT-07:00 Sergey Beryozkin <sberyozkin@gmail.com>:
>
>> Hi All,
>>
>> One thing which is interesting here is that in case of WebClient, if the
>> auto-closure is disabled, it is the responsibility of the client code to do
>> Response.close
>>
>> JAX-RS API allows to do something like
>> InputStream is = client.get(InputStream.class)
>>
>> or
>>
>> Response r = client.get();
>> InputStream is = r.readEntity(InputStream.class);
>>
>> As I said earlier, the auto-closure only works reliably if we have a case
>> of the complete deserialization, ex:
>>
>> Book b = client.get(Book.class);
>>
>> but is not so in cases like
>>
>> Document d = client.get(Document.class);
>> due to the fact DOM can be lazily initialized
>> or
>> Source jaxpSource = client.get(Source.class);
>>
>> So you need to enable the input stream auto-closure or get Response from
>> (Web)Client (casting a proxy to it if needed) and calling close on that...
>>
>> Cheers. Sergey
>>
>>
>> On 05/08/15 18:10, Jose María Zaragoza wrote:
>>
>>> 2015-08-05 18:55 GMT+02:00 Daniel Kulp <dkulp@apache.org>:
>>>
>>>>
>>>> On Aug 5, 2015, at 12:04 PM, Jose María Zaragoza <demablogia@gmail.com>
>>>>> wrote:
>>>>>
>>>>>
>>>>> I'm trying to understand how is the high-level behaviour ( client side,
>>>>> so far )
>>>>> Please, let me know if I'm wrong ( I'm sure about that )
>>>>>
>>>>> 1) I guess that a WebClient instance uses a new HttpURLConnection per
>>>>> thread/IO operation ( and executes openConnection() every time )
>>>>> 2) I guess that HttpURLConnection can reuses a TCP connection
>>>>>
>>>>
>>>> Correct.
>>>>
>>>> 3) I guess that HttpConduit manages HttpURLConnection life-cycle and I
>>>>> guess that, if TCP connection is closed *by server/firewall*  ( sends
>>>>> FIN packet ), it closes underlying TCP connection ( I don't know how
>>>>> either HttpURLConnection.disconnect() or instream.close()
>>>>>
>>>>
>>>> Well, not really.   We call instream.close() and that’s about it.   The
>>>> HttpURLConnection stuff built into the JDK takes care of all the TCP
>>>> related things as well as the keep-alive stuff and such.   We don’t even
>>>> have access to the TCP socket stuff to do much with it.
>>>>
>>>
>>>
>>> Thanks
>>> Basically I would like to know if CXF handles in any way the closed
>>> connections by remote server/firewall .
>>> If you tell me that , at least, CXF calls instream.close(), it's enough
>>> for me
>>>
>>> I know that underlying TCP connections/sockets are managed by
>>> HttpURLConnection internally, but I would like what manages
>>> HttpURLConnection
>>>
>>> Regards
>>>
>>>
>>>
>>>>
>>>> My questions are about what happens if a firewall closes a TCP
>>>>> connection , established between my WebClient and a server
>>>>> My client should close underlying TCP connection but that is managed
>>>>> by HttpConduit , right ?
>>>>>
>>>>
>>>> No.   By the internals of the HttpURLConnection stuff in the JDK.   When
>>>> we call the “openConnection()” method of the HttpURLConnection, internally
>>>> it looks at the information we have set on the HttpURLConnection (host,
>>>> port, etc..) and grabs a “com.sun.*.Something” from it’s internal
>>>> connection pool and the HttpURLConnection then uses whatever that is for
>>>> the communication.  It’s not something we really have any control over.
>>>>
>>>> Dan
>>>>
>>>>
>>>>
>>>>
>>>>> Maybe I'm wrong at all , sorry
>>>>>
>>>>> Regards
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> Cheers, Sergey
>>>>>>
>>>>>> On 05/08/15 13:33, Jose María Zaragoza wrote:
>>>>>>
>>>>>>>
>>>>>>> 2015-08-05 12:08 GMT+02:00 Sergey Beryozkin <sberyozkin@gmail.com>:
>>>>>>>
>>>>>>>>
>>>>>>>> Hi
>>>>>>>> On 04/08/15 20:26, Jose María Zaragoza wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 2015-08-02 23:12 GMT+02:00 Sergey Beryozkin <sberyozkin@gmail.com>:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Hi
>>>>>>>>>> Can you cast a proxy to WebClient (via its utility
method
>>>>>>>>>> WebClient.client()) and close it, and also enable
the auto-closure
>>>>>>>>>> of
>>>>>>>>>> the
>>>>>>>>>> Response ?
>>>>>>>>>> Sergey
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Hi
>>>>>>>>>
>>>>>>>>> I'm using CXF 2.7.8 to invoke REST services by WebClient
API &
>>>>>>>>> JAX-RS
>>>>>>>>> Client Proxy-based API
>>>>>>>>> Is required to call explicitly close() method in WebClient
?
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Calling close() helps with the proactive conduit (HTTP client
state)
>>>>>>>> cleanup, it is a good idea to call close()
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Anyway I would like to keep alive some TCP connection
( default
>>>>>>>>> value
>>>>>>>>> ) . I wonder in which cases i should set auto-closure
or call
>>>>>>>>> explicitly close()
>>>>>>>>>
>>>>>>>>> The auto-closure can only be applied to JAX-RS Response
streams, in
>>>>>>>> cases
>>>>>>>> where you have a code like
>>>>>>>> Book book = proxy.getBook();
>>>>>>>> or
>>>>>>>> Book book = webClient.get().readEntity(Book.class);
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>> My code looks like
>>>>>>>
>>>>>>> WebClient client;
>>>>>>>
>>>>>>> try
>>>>>>> {
>>>>>>> javax.ws.rs.core.Response r = this.client.path("/users/{phone}/id",
>>>>>>> phone).get();
>>>>>>> IdGETResponseType idGETResponseType = (IdGETResponseType)
>>>>>>> r.readEntity(IdGETResponseType.class);
>>>>>>> ...
>>>>>>> ....
>>>>>>> }
>>>>>>> finally
>>>>>>> {
>>>>>>> // Clean
>>>>>>> if (client != null)
>>>>>>> client.reset();
>>>>>>> }
>>>>>>>
>>>>>>> Should I perform a explicit close on WelcClient object  ? What
happens
>>>>>>> if I dont ?
>>>>>>>
>>>>>>>
>>>>>>> WebClient is injected  by Spring container
>>>>>>>
>>>>>>>    <jaxrs:client id="client"
>>>>>>>            address="${api.endpoint}"
>>>>>>>            serviceClass="org.apache.cxf.jaxrs.client.WebClient"
>>>>>>>            threadSafe="true">
>>>>>>>            <jaxrs:headers>
>>>>>>>             <entry key="Content-Type"
>>>>>>> value="application/json;charset=UTF-8"/>
>>>>>>>                <entry key="Accept" value="application/json"/>
>>>>>>>            </jaxrs:headers>
>>>>>>>            <jaxrs:providers>
>>>>>>>       <ref bean="jsonProvider"/>
>>>>>>>     </jaxrs:providers>
>>>>>>>
>>>>>>>     </jaxrs:client>
>>>>>>>
>>>>>>> Regards
>>>>>>>
>>>>>>>
>>>>>>>> we've had side-effects with the Response stream auto-closures
in
>>>>>>>> cases
>>>>>>>> like
>>>>>>>>
>>>>>>>> Document domDoc = webClient.get().readEntity(Document.class);
>>>>>>>>
>>>>>>>> WebClient was also auto-closed in 3.1.1 which was reverted
in 3.2.2
>>>>>>>> because
>>>>>>>> when WebClient is copied and one of these clients gets out
of scope
>>>>>>>> then
>>>>>>>> its
>>>>>>>> (shared) http conduit gets closed and the other client gets
broken. I
>>>>>>>> might
>>>>>>>> revisit it and make sure the conduit is cloned in such cases,
but not
>>>>>>>> right
>>>>>>>> now...
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I'm already calling reset() method for cleaning thread-local
info (
>>>>>>>>> I
>>>>>>>>> guess  )
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> if you intend to share the client between multiple threads
then it
>>>>>>>> should
>>>>>>>> be
>>>>>>>> up to Spring/etc to close it when the context goes out of
scope
>>>>>>>>
>>>>>>>> Cheers, Sergey
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On 31/07/15 02:59, Xiaobin wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Hi guys,
>>>>>>>>>>>
>>>>>>>>>>> I am using CXF 2.7.5. I have an application using
>>>>>>>>>>> JAXRSClientFactoryBean
>>>>>>>>>>> to
>>>>>>>>>>> generate proxy, and use following code to close
client when it is
>>>>>>>>>>> done:
>>>>>>>>>>>
>>>>>>>>>>> ClientConfiguration config = WebClient.getConfig(root);
>>>>>>>>>>>         HTTPConduit conduit = config.getHttpConduit();
>>>>>>>>>>>         if (conduit == null) {
>>>>>>>>>>>           throw new IllegalArgumentException();
>>>>>>>>>>>         }
>>>>>>>>>>>         conduit.close();
>>>>>>>>>>>
>>>>>>>>>>> As time goes on, I noticed that there are many
connections shown
>>>>>>>>>>> by
>>>>>>>>>>> netstat
>>>>>>>>>>> in CLOSE_WAIT state.
>>>>>>>>>>>
>>>>>>>>>>> I understand that because of CXF-5144, it won't
be able to re-use
>>>>>>>>>>> connections. But besides this, is there anything
I can do with
>>>>>>>>>>> those
>>>>>>>>>>> CLOSE_WAIT connections ? Are these going to time
out eventually
>>>>>>>>>>> or ?
>>>>>>>>>>>
>>>>>>>>>>> Also, I am wondering if setting ConnectionType.CLOSE
would help ?
>>>>>>>>>>>
>>>>>>>>>>> Look forward to your suggestions! Thanks in advance.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> -Xiaobin
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Sergey Beryozkin
>>>>>>>>
>>>>>>>> Talend Community Coders
>>>>>>>> http://coders.talend.com/
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Sergey Beryozkin
>>>>>>
>>>>>> Talend Community Coders
>>>>>> http://coders.talend.com/
>>>>>>
>>>>>
>>>> --
>>>> Daniel Kulp
>>>> dkulp@apache.org - http://dankulp.com/blog
>>>> Talend Community Coder - http://coders.talend.com
>>>>
>>>>
>>
>> --
>> Sergey Beryozkin
>>
>> Talend Community Coders
>> http://coders.talend.com/
>>
>
>
>


Mime
View raw message