axis-java-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dobri Kitipov <kdobrik.ax...@googlemail.com>
Subject Re: HTTP connection leak and other related issues
Date Fri, 27 Feb 2009 09:10:28 GMT
Hi all,
I have observed absolutely the same thing these days. I need some more time
to analyze the whole picture, but here is my current synthesis of the issue.

It seems that http connection release is tightly coupled  with the Axis2
builder used to handle and process the response body and the corresponding
input streams used. The builder and the InputStream used are based on the
HTTP headers fields like "Content-Type" and "Transfer-Encoding" (e.g.
Content-Type: text/xml; charset=UTF-8  Transfer-Encoding: chunked). So if we
have Content-Type: text/xml; then SOAPBuilder class will be used. If we
have  type="application/xop+xml" then MIMEBuilder will be used.

The successfull story when we have MIMIBuilder:

When MIMEBuilder is used then the response Buffered InputStream (IS) is
wrrapped (I will use "->" sign as substitution for wrrapped) ChunkedIS ->
AutoCloseIS (this has a responseConsumed() method notified when IS.read()
returns -1, which means that the response IS has been completely read) ->
PushBackIS ->BounderyPushBackIS. The BounderyPushBackIS reads the response
stream (see readFromStream(....)) in a cycle till it reaches its end. At
every iteration of this cycle a AutoCloseIS checkClose(l) is invoked. So
when the end is reached (-1 is returned) then this check causes the
invokation of the AutoCloseIS checkClose(...)  method. This method invokes
notifyWatcher() that in turn invokes responseBodyConsumed() method of the
HttpMethodBase class. This causes the release of the http connection which
is returned back to the connection pool. So here we have no problem with
connection reuse.

The bad story we have with SOAPBuilder:

When SOAPBuilder is used then the response Buffered InputStream is wrrapped
in a ChunkedIS -> AutoCloseIS -> PushBackIS. May be you has noticed that
BounderyPushBackIS is not used. As a result the respose IS is not completely
read (in fact this is not really correct, it could be read but the
invokation of the PushBackIS unread(...) method causes the AutoCloseIS
checkClose() method never to return -1). As a result the http connection is
not released. And since there is a limit to have 2 connection per host
then after the second invokation of the WS client the thread hangs waiting
for a free connection.

Please, provide us with your comments on this issue.

Thank you in advance.

Regards,
Dobri

On Fri, Feb 27, 2009 at 3:54 AM, Alexis Midon <midon@intalio.com> wrote:

> no taker for an easy patch?
>
> Alexis
>
>
> On Wed, Feb 25, 2009 at 6:45 PM, Alexis Midon <midon@intalio.com> wrote:
>
>>
>> Hi everyone,
>>
>> All the issues relatives to AXIS2-935 are really messy, some of them are
>> closed but their clones are not. Some are flagged as fixed but are obviously
>> not. All these issues are really old, so I'd like to take a chance to bring
>> them back to your attention, especially before releasing 1.5.
>>
>> I'll post a description of the issue in this email as a summary all the
>> jiras.
>>
>> By default, ServiceClient uses one HttpConnectionManager per invocation
>> [2]. This connection manager will create and provide one connection to
>> HTTPSender. The first issue is that by default this connection is never
>> released to the pool [3]. if you do zillions of invocations, this leak will
>> max out your number of file descriptors.
>>
>> Your investigations in Axis2 options quickly lead you to the
>> REUSE_HTTP_CLIENT option. But this first issue has some unfortunate
>> consequences if you activate it. Actually if you do so, a single connection
>> manager is shared across all invocations. But because connections are not
>> release, the pool is starved after two invocations, and the third invocation
>> hangs out indefinitely. :(
>>
>> If you keep digging you will find the AUTO_RELEASE_CONNECTION option. Its
>> sounds like a good lead! Let's try it. If you activate this option the
>> connection is properly released -Yahoooo! the leak is fixed - but
>> unfortunately a new issue shows up (issue #2, aka AXIS2-3478).
>> AbstractHTTPSender passes the stream of the connection to the message
>> context [4] , but that the connection is now properly released, so this
>> stream is closed before the SOAPBuilder gets a chance to read the response
>> body.
>> Boom! "IOException: Attempted read on closed stream"
>>
>> These issues are easily reproducible in versions 1.3, 1.4, 1.5.
>>
>> I submitted and documented a fix in AXIS2-2931 [5], if you had a chance to
>> look at it that would be much appreciate.
>>
>>
>> Alexis
>>
>>
>> [1]
>> https://issues.apache.org/jira/browse/AXIS2-935?focusedCommentId=12513543#action_12513543
>> [2] see method getHttpClient in
>> https://svn.apache.org/repos/asf/webservices/commons/trunk/modules/transport/modules/http/src/org/apache/axis2/transport/http/AbstractHTTPSender.java
>> [3] see method cleanup in
>> https://svn.apache.org/repos/asf/webservices/commons/trunk/modules/transport/modules/http/src/org/apache/axis2/transport/http/HTTPSender.java
>> [4] see method processResponse in AbstractHTTPSender.java
>> [5]
>> https://issues.apache.org/jira/browse/AXIS2-2931?focusedCommentId=12676837#action_12676837
>>
>
>

Mime
View raw message