Mailing-List: contact httpclient-users-help@hc.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "HttpClient User Discussion" <httpclient-users@hc.apache.org>
Received-SPF: pass (athena.apache.org: local policy includes SPF record at
 spf.trusted-forwarder.org)
Message-ID: <1395740683.10953.12.camel@ubuntu>
Subject: Re: Connection leakage from Migrating 4.2.3 -> 4.3.3
From: Oleg Kalnichevski <olegk@apache.org>
To: HttpClient User Discussion <httpclient-users@hc.apache.org>
Date: Tue, 25 Mar 2014 10:44:43 +0100
In-Reply-To: 
 <CAJZgJGJfgdRdU28aqtrO2LpuoF1V90-37RrVB1cpmfETcGRszg@mail.gmail.com>
References: 
	<CAJZgJGJfgdRdU28aqtrO2LpuoF1V90-37RrVB1cpmfETcGRszg@mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit

On Mon, 2014-03-24 at 21:09 -0400, Nick Chang wrote:
> Hello,
> 
> We are using httpclient to build a web proxy application in our product. We
> use 4.2.3 version and it has been working flawlessly. Now I am trying to
> migrate the product's httpclient library from 4.2.3 to 4.3.3 to achieve
> SNI-Extension support with JDK1.7. We have to re-factor several deprecated
> methods and classes in 4.2.3, such as changing
> PoolingClientConnectionManager to PoolingHttpClientConnectionManager,
> changing DefaultHttpClient to CloseableHttpClient.
> 
> However, I am running into an issue that connection is not being released
> and eventually execute() got the exception in waiting for a connection
> (Exception Text:org.apache.http.conn.ConnectionPoolTimeoutException:
> Timeout waiting for connection from pool).
> 
> Below is the code sniplet in a worker thread.
> 
> The 4.2.3 code
> 
>         m_httpClient = new DefaultHttpClient(m_connectionManager, params);
>         ...
>         m_httpResponse = m_httpClient.execute(m_httpCommand, localContext);
>         ...
>         BasicManagedEntity entity =
> (BasicManagedEntity)m_httpResponse.getEntity();
>         if (entity!= null) {
>             m_inputStream = entity.getContent();
>         }
>         .. process inputStream ...
>         if (m_inputStream != null) {
>             m_inputStream.close();
>         }
> 
>         m_httpClient.getConnectionManager().closeIdleConnections(5,
> TimeUnit.SECONDS);
> 
> The 4.3.3 code
>         ...
>         m_httpClient = HttpClients.custom()
>             .setConnectionManager(m_connectionManager)
>             .setDefaultCredentialsProvider(credentialsProvider)
>             .setDefaultRequestConfig(m_defaultRequestConfig)
>             .disableCookieManagement()
>             .disableRedirectHandling()
>             .disableAutomaticRetries()
>             .setKeepAliveStrategy(keepAliveStrategy)
>             .build();
>         ...
>         m_httpResponse = m_httpClient.execute(m_httpCommand);
>         ...
>         m_entity = (HttpEntity)m_httpResponse.getEntity();
>         if (m_entity!= null) {
>               m_inputStream = m_entity.getContent();
>         }
>         .. process inputStream ...
>         try {
>             EntityUtils.consume(m_entity);
>         } catch (IOException e) {
>             // TODO Auto-generated catch block
>             e.printStackTrace();
>         }
>     ...
>         try {
>             if (m_inputStream != null) {
>                 m_inputStream.close();
>             }
> 
>         } catch (IOException e) {
>             // TODO Auto-generated catch block
>             e.printStackTrace();
>         }
>     ...
>         try {
>             m_httpResponse.close();
>         } catch (IOException e) {
>             // TODO Auto-generated catch block
>             e.printStackTrace();
>         }
> 
> (I've followed the instruction in this email.
> http://mail-archives.apache.org/mod_mbox/hc-httpclient-users/201211.mbox/%3C1353079080.20463.12.camel%40ubuntu%3Eto
> close the connection.)
> 
> The 4.2.3 code works flawlessly. The number of leased connection always
> goes back to zero. But in the 4.3.3 code, the number of leased connection
> gradually increased.
> 
> I added IdleConnectionMonitorThread class as in
> http://hc.apache.org/httpcomponents-client-4.3.x/tutorial/html/connmgmt.html#d5e405It
> did help reducing the number of leased
> connection, but the number of leased connections still gradually increased.
> 
> Hence, here are my questions:
> 
> 1. How would I track down this connection leakage?

Connection context logging would probably be what I would use facing the
same problem. See section on context logging for connection management /
request execution:

http://hc.apache.org/httpcomponents-client-4.3.x/logging.html

> 2. Is there a suggestion or best practice to migrate the deprecated classes
> and methods in 4.2.3 to the current ones in 4.3.3?

There is no migration guide as such. HttpClient tutorial should be the
most complete source of information on HttpClient best practices.

http://hc.apache.org/httpcomponents-client-4.3.x/tutorial/html/index.html

> 3. Is a separate thread, IdleConnectionMonitorThread, required to clean up
> the leased connection in 4.3.3? If so, why is it not needed in 4.2.3?
> 

IdleConnectionMonitorThread does not touch leased connections in any
form or fashion. It works with available (idle) connections. It has no
relevance for connection lease / release cycle, which is entirely in the
consumer's responsibility.

Oleg


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org