hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From George Peristeris <tsinan...@yahoo.gr>
Subject Too many connections in FIN_WAIT_2 state
Date Thu, 16 Dec 2004 15:26:44 GMT
Hi all,

HttpLient 3.0 beta1
Java 1.4.1_06
Solaris 8 2/02 s28s_u7wos_08a SPARC

I am using the MultiThreadedHttpConnectionManager and albeit I set 
both DefaultMaxConnectionsPerHost and MaxTotalConnections, whenever 
the server I am hitting crashes I see too many TCP sockets in 
FIN_WAIT_2 state.
"Too many" meaning many more than the limits I have set.

I am using httpclient from within a thread to perform a simple GET on

a remote host.I instantiate MultiThreadedHttpConnectionManager and 
HttpClient once , I set the limits for my connections and the
timeouts
and from within its thread's run() I create a new GetMethod which I 
execute using the instance of HttpClient and then I always 
releaseConnection(). Currently , since I am still testing, I do read 
the response but when this goes live I wouldn't mind not reading it
at 
all (in fact I 'd rather not, since all the info I need is the 
response status code).


To sum it up:
1. Does the limit set by MaxTotalConnections(int) guarantee the 
maximum number  of sockets opened by the corresponding 
MultiThreadedHttpConnectionManager?
  -If so can you come up with an explanation of the issue I am
facing?
 (max=40 , netstat shows more than 250 sockets in FIN_WAIT_2 
 whenever the remote host crashes , that is, it is up and runing and
 due to the load I send at some point it crashes and obviously does 
 not send back  the FIN packet the socket on my side is waiting for ,
 so it stays there till it times out.)
 My problem is NOT that the socket is in FIN_WAIT_2, but the fact
that 
 the number of the sockets exceeds the limit I have set. If this is 
 the case my application will eventually run out of file descriptors.
 I don't mind If some threads cannot perform the GET but I do mind if

 the whole application crashes.

  -If not (in this case I have misunderstood what MaxTotalConnections

   is, and a brief explanation would be highly appreciated ) any 
   ideas/hints on how this functionality may be applied are welcome.
  I have thought of adding header "Connection : close", which from 
  what I have read in other threads may solve the problem, but I
think 
  this cancels the whole concept of connection reuse and propably 
  slows things down(?).


2. How would NOT reading the response body (but calling 
   releaseConnection() after each executeMethod() ) affect the whole 
   connection management thing? 

  This is the exception thrown when the remote server crashes and 
  FIN_WAIT_2 sockets start pilling up on my side

 
java.net.SocketTimeoutException: Read timed out
        at java.net.SocketInputStream.socketRead0(Native Method)
        at
java.net.SocketInputStream.read(SocketInputStream.java:129)
        at
java.io.BufferedInputStream.fill(BufferedInputStream.java:183)
        at
java.io.BufferedInputStream.read(BufferedInputStream.java:201)
        at
org.apache.commons.httpclient.HttpParser.readRawLine(HttpParser.java:76)
        at
org.apache.commons.httpclient.HttpParser.readLine(HttpParser.java:104)
        at
org.apache.commons.httpclient.HttpConnection.readLine(HttpConnection.java:1112)
        at
org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.readLine(MultiThreadedHttpConnectionManager.java:1379)
        at
org.apache.commons.httpclient.HttpMethodBase.readStatusLine(HttpMethodBase.java:1825)
        at
org.apache.commons.httpclient.HttpMethodBase.readResponse(HttpMethodBase.java:1588)
        at
org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:999)
        at
org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:382)
        at
org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:168)
        at
org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:393)
        at
org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:324)
        at
com.vodafone.sdp.global.access.authentication.spi.provider.GrCurAccessThread.run(GrCurAccessThread.java:138)
        
GrCurAccessThread.java:138 -> CurResponseStatusCode =
CurHttpClient.executeMethod(CurGetMethod);


Sample Code:

public class GrCurAccessThread extends Thread {
         .
         . 
         .
        static MultiThreadedHttpConnectionManager
multiThreadedHttpConnectionManager = new
MultiThreadedHttpConnectionManager();
        static HttpClient CurHttpClient = new
HttpClient(multiThreadedHttpConnectionManager);
        static {
          //HttpClient 3.0 API
          
multiThreadedHttpConnectionManager.getParams().setDefaultMaxConnectionsPerHost(40);
          
multiThreadedHttpConnectionManager.getParams().setMaxTotalConnections(40);

           //set all timeouts
          
multiThreadedHttpConnectionManager.getParams().setConnectionTimeout(HTTP_CONNECTION_TIMEOUT);
          
CurHttpClient.getParams().setConnectionManagerTimeout(HTTP_CONNECTION_FACTORY_TIMEOUT);
          
multiThreadedHttpConnectionManager.getParams().setSoTimeout(HTTP_SOCKET_TIMEOUT);
        }
        .
        .
        .
   public GrCurAccessThread(...){...}

	public void run(){
	.
	.
	.
	GetMethod CurGetMethod = new GetMethod(GR_CUR_HTTP_API_URL);

        int CurResponseStatusCode = -1 ;

        CurGetMethod.setRequestHeader("my_header1",my_header1_value);
        CurGetMethod.setRequestHeader("my_header2",my_header2_value);
        
        CurGetMethod.setFollowRedirects(false);

        try {
            CurResponseStatusCode =
CurHttpClient.executeMethod(CurGetMethod);
            
           if(logger.isDebugEnabled()){
              String tmp =
(CurGetMethod.getResponseHeader("Content-Length")).getValue();
             
logger.debug("getResponseHeader(\"Content-Length\").getValue() = " +
tmp);
              logger.debug("getResponseBody()=" + new
String(CurGetMethod.getResponseBody())); //}
              Header[] headers = CurGetMethod.getResponseHeaders();
              for (int i = 0; i < headers.length; i++) {
                logger.debug("Response header[" + i + "] :=="
+headers[i].toString());
              }
            }
        .
        .
        .logger.info(....);
        } catch (HttpException recex) { //we actually don't need to
catch this since we won't retry (HttpRecoverableException deprecated)
            .
            .
        } catch (IOException ioex) {...
        } catch (Exception ex){...
        } finally{
          CurGetMethod.releaseConnection();
        .
        .
        . }
        }
}

 thanx for your effort,
 
 george peristeris


P.S> I have seen that the httpClient 3.0 also needs
commons-codec-1.3, and I think it's not mentioned.

____________________________________________________________
Do You Yahoo!?
Αποκτήστε τη δωρεάν @yahoo.gr διεύθυνση σας στο http://www.otenet.gr

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org


Mime
View raw message