hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sebb <seb...@gmail.com>
Subject Re: BindException while using HttpClient under load
Date Mon, 24 May 2010 23:29:08 GMT
On 25/05/2010, CrystalCracker <purpleheart36@hotmail.com> wrote:
>
>  I have got 1000 dedicated Java threads where each thread polls a
>  corresponding url every one second.
>
>  Simplified version of the code is like this
>
>  MyThread extends() {
>    Node node;
>    MyThread(Node node) {
>       this.node = node;
>    }
>    public void run() {
>        Poller.poll()
>    }
>  }
>
>  public class Poller {
>     public static Node poll(Node node) {
>         GetMethod method =  null;
>         try {
>             HttpClient client = new HttpClient(new
>  SimpleHttpConnectionManager(true));
>             client.getParams().setParameter("http.socket.timeout", 120000);
>
>             method = new GetMethod(node.getUrl());
>             method.getParams().setSoTimeout(8000);
>
>             client.executeMethod(method);
>
>             method.getResponseHeaders();
>             method.getResponseBody();
>         } catch (IOException ex) {
>             ex.printStackTrace();
>         } finally {
>             method.releaseConnection();
>         }
>     }
>  }
>
>  The threads are run one second:
>
>  for (int i=0; i <1000; i++) {
>     MyThread thread = threads.get(i) // threads  is a static field
>     if(thread.isAlive()) {
>         // If the previous thread is still running, let it run.
>     } else {
>         thread.start();
>     }
>  }
>
>  The problem is if I run the job every one second I get random exceptions
>  like these:
>
>  java.net.BindException: Address already in use

In spite of what the Exception says, I've seen it sometimes when the
host has run out of sockets.

>   INFO httpclient.HttpMethodDirector: I/O exception (java.net.BindException)
>  caught when processing request: Address already in use
>   INFO httpclient.HttpMethodDirector: Retrying request
>
>  But if I run the job every 2 seconds or more, everything runs fine.
>
>  I even tried shutting down the instance of SimpleHttpConnectionManager()
>  using shutDown() with no effect.
>
>  It looks like if when myThread has finished running, the HttpClient it
>  started hasn't shutdown even though I told it to release conenction and/or
>  shutdown the manager.
>
>  Any ideas?

Might be due to the timeouts inherent in shutting down a TCP socket -
the OS has to wait to ensure that there aren't still some packets
around somewhere. At the slower rate, the OS has enough time to
cleanup, but not at the faster rate.

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org


Mime
View raw message