hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ken DeLong" <ken.del...@babycenter.com>
Subject RE: Too Many Connections
Date Fri, 27 Jun 2008 18:25:00 GMT
All of our connections are to the exact same host - in fact, the sending and receiving processes
are on the same actual server.  The same dns name is used for each request (we use the dns
name of the machine, not the string "localhost").

We'll try the multithreaded connection manager next.  (I was pretty sure the calling app was
single threaded but I'm not 100% sure).

Thanks!
Ken


-----Original Message-----
From: Oleg Kalnichevski [mailto:olegk@apache.org] 
Sent: Friday, June 27, 2008 11:20 AM
To: HttpClient User Discussion
Subject: RE: Too Many Connections

On Fri, 2008-06-27 at 09:08 -0700, Ken DeLong wrote:
> That didn't help.  We changed to
> 
> 	private HttpClient client = new HttpClient(new 
> SimpleHttpConnectionManager());
> 

´╗┐Ken,

´╗┐SimpleHttpConnectionManager re-uses the underlying connection object but it cannot reuse
the network socket when connecting to a different host. Ii needs to close the old socket and
open a new one every time it connects to a different host. If you would like to keep alive
connections to multiple hosts, consider using ´╗┐MultiThreadedHttpConnectionManager

Regarding connections in TIME_WAIT state please see

http://wiki.apache.org/HttpComponents/FrequentlyAskedConnectionManagementQuestions

Oleg


> But we still see the same behavior.
> 
> Turning on the debug logs shows messages like
> 
> $ fgrep 'Connection is locked.  Call to releaseConnection() ignored.'
> server.log | wc -l
> 176
> 
> $ fgrep 'Releasing connection back to connection manager.' server.log 
> | wc -l
> 264
> 
> 
> What would cause the connection to be locked when releaseConnection() 
> is called?
> 
> 
> 
> -----Original Message-----
> From: Pete Keyes [mailto:PKeyes@starbucks.com]
> Sent: Thursday, June 26, 2008 3:27 PM
> To: HttpClient User Discussion
> Cc: Stephen Hiley
> Subject: RE: Too Many Connections
> 
> You need to use either of these:
>   - org.apache.commons.httpclient.MultiThreadedHttpConnectionManager
>   - org.apache.commons.httpclient.SimpleHttpConnectionManager
> There are examples of both in the HttpClient site.
> 
> In addition to that you may need to tune the O/S to formally release 
> the CLOSE_WAIT files sooner.  There is a delay between when an 
> application closes a socket and when the O/S can formally release that 
> socket to avoid orphan packets.
> 
> See the documentation for your O/S on setting that value.
> 
>  
> ...Pete
> Starbucks Coffee Co.
> 2601 Utah Ave S.
> Seattle, WA. 98134
> (w) 206-318-5933
> (c) 206-914-4134
> 
> -----Original Message-----
> From: Ken DeLong [mailto:ken.delong@babycenter.com]
> Sent: Thursday, June 26, 2008 3:17 PM
> To: httpclient-users@hc.apache.org
> Cc: Stephen Hiley
> Subject: Too Many Connections
> 
> We have a method in our app that calls httpclient like this:
> 
> 	private HttpClient client = new HttpClient();
> 	
> 	public String send(String urlString, String xml)
> 	{
> 		PostMethod postMethod = new PostMethod(urlString);
> 		
> 		// Put the XML in the request
> 		try
> 		{
> 			RequestEntity entity = new
> StringRequestEntity(xml, "text/xml", "UTF-8");
> 			postMethod.setRequestEntity(entity);
> 		}
> 		catch(UnsupportedEncodingException e1)
> 		{
> 			logger.warn("Couldn't encode the xml?", e1);
> 			return "" +
> IContentPublishingService.ACTION_ERROR;
> 		}
> 
> 		// Send the data and get the response
> 		String response = null;
> 		try
> 		{
> 			client.executeMethod(postMethod);
> 			byte[] responseBody =
> postMethod.getResponseBody();
> 			response = new String(responseBody);
> 		}
> 		catch(HttpException e)
> 		{
> 			logger.warn("Fatal protocol violation: " + e.getMessage());
> 			return "" +
> IContentPublishingService.ACTION_ERROR;		
> 		}
> 		catch(IOException e)
> 		{
> 			logger.warn("Fatal transport error: " + e.getMessage());
> 			return "" +
> IContentPublishingService.ACTION_ERROR;
> 		}
> 		finally
> 		{
> 			// Release the connection.
> 			postMethod.releaseConnection();
> 		}
> 
> 		return response;
> 	}
> 
> 
> This method is called rapidly by a single thread in our application.
> After 1500 calls or so, we start to see java.net.SocketException, too 
> many open files.  Netstat reveals that there are many sockets in 
> CLOSE_WAIT and TIME_WAIT states.  It seems that httpclient is not 
> reusing the connection managed by the HttpClient instance, and instead 
> is closing the connection and creating a new one.  This appears to be 
> in conflict with all the documentation that we could find.
> 
> An interesting wrinkle is that the sending and receiving side of the 
> connection are both on the same machine.  The URL is constructed using 
> the machine's dns name, not "localhost".
> 
> ----------------------------------------------------------------------
> --
> --
> Kenneth DeLong
> Software Architect
> Engineering
> BabyCenter, LLC
> 
> kdelong@babycenter.com
> p:  415.344.7616
> AIM: kenwdelong
> 
> http://www.babycenter.com
> 
> 
> 
> This email message is for the sole use of the intended recipient(s) 
> and may contain confidential and privileged information. Any 
> unauthorized review, use, disclosure or distribution is prohibited. If 
> you are not the intended recipient, please contact the sender by reply 
> email and destroy all copies of the original message. If you are the 
> intended recipient, please be advised that the content of this message 
> is subject to access, review and disclosure by the sender's Email 
> System Administrator.
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org

Mime
View raw message