hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brad <b...@javawork.co.uk>
Subject Re: Too many open sockets
Date Fri, 16 Nov 2007 16:50:10 GMT
Oleg,

a very good point, I hadn't considered that the server could be
closing the connection.

Thanks for you patience.

On Nov 16, 2007 4:38 PM, Oleg Kalnichevski <olegk@apache.org> wrote:
>
> On Fri, 2007-11-16 at 16:29 +0000, Brad wrote:
> > Oleg,
> >
> > once again thanks for taking the time to respond.
> >
> > What I don't understand is that if it reuses sockets by default and
> > I'm not calling anything to close them, why is it running out? I'm
> > calling recycle (to reuse the same http connection) and yet it
> > consistently runs out of sockets around the 3200 iteration mark. To my
> > untrained eye, it looks like its using up every available port between
> > 1024 and 4999 which seems to be the highest port number windows is
> > willing to assign to it.
> >
> > I do agree that it would be impossible for it to re-use closed
> > sockets. What I don't understand is why they are being closed in the
> > first place. If commons-http is using pooling by default and I'm
> > calling recycle(), theoretically shouldn't it be able to execute http
> > methods indefinitely?
> >
> > I expect I'm missing something here and am sure I'll have an "oh
> > yeah!" moment any time soon!
> >
>
> Brad,
> It takes two to tango. Both endpoints, client and server, need to be (1)
> able and (2) willing to keep the connection alive for it to be
> persistent between requests. It can well be the target server. You can
> examine HttpClient wire/context logs to see exactly why connections get
> closed in the first place.
>
> http://jakarta.apache.org/httpcomponents/httpclient-3.x/logging.html
>
> Hope this helps
>
> Oleg
>
>
>
> > Thanks,
> > Brad.
> >
> > On Nov 16, 2007 12:47 PM, Oleg Kalnichevski <olegk@apache.org> wrote:
> > >
> > > On Fri, 2007-11-16 at 11:25 +0000, Brad wrote:
> > > > Oleg,
> > > >
> > > > thanks for your reply.
> > > >
> > > > So if I understand correctly, what that is implying in this situation
> > > > is that there is a limit to how many sockets can be opened and closed
> > > > in a given time period?
> > > >
> > >
> > > Brad,
> > >
> > > (1) As far as JVM is concerned sockets in the TIME_WAIT state are
> > > considered closed. Those sockets are kept active for a while by the
> > > underlying OS in order to catch stray packets after the connection has
> > > been closed by the JVM process.
> > >
> > > You need to tweak TCP/IP settings of the OS you are using if you want to
> > > reduce the period of time sockets are kept in the TIME_WAIT state.
> > >
> > >
> > > > If that's the case, then fair enough but I was hoping there would  be
> > > > some way to make commons-http re-use a socket. That seemed to be the
> > > > whole point of the recycle() method. The documentation seemed to be
> > > > suggesting that it would use keep-alives to avoid opening a new socket
> > > > every time. Is there any way around this?
> > > >
> > >
> > > HttpClient can re-use connections and does that per default [1], but it
> > > obviously cannot re-use closed sockets.
> > >
> > > Hope that helps
> > >
> > > Cheers,
> > >
> > > Oleg
> > >
> > > [1]
> > > http://jakarta.apache.org/httpcomponents/httpclient-3.x/performance.html#Connection_persistence
> > >
> > >
> > >
> > > > Brad.
> > > >
> > > > On Nov 16, 2007 11:03 AM, Oleg Kalnichevski <olegk@apache.org> wrote:
> > > > >
> > > > >
> > > > > On Fri, 2007-11-16 at 10:54 +0000, Brad wrote:
> > > > > > Hi,
> > > > > >
> > > > > > I'm using http-commons to make some calls to a http server.
There are
> > > > > > situations where I may have to make a large volumen of calls
in a
> > > > > > short period of time.
> > > > > >
> > > > > > My problem is that http-commons seems to be eating connections,
> > > > > > resulting in eventually getting this error
> > > > > >
> > > > > >           java.net.BindException: Address already in use: connect
> > > > > >
> > > > > > A quick netstat from the command prompt shows thousands of sockets
> > > > > > with status TIME_WAIT.
> > > > > >
> > > > > > Here's my simple test code:
> > > > > >
> > > > > >         public void testHttpGet() {
> > > > > >
> > > > > >                 HttpClient client = new HttpClient();
> > > > > >                 GetMethod get = new GetMethod();
> > > > > >
> > > > > >                 try {
> > > > > >
> > > > > >                         for (int i = 0; i < 10000; i++) {
> > > > > >
> > > > > >                                 get.setPath("http://localhost:8081/");
> > > > > >                                 int code = client.executeMethod(get);
> > > > > >                                 byte[] responseBody =  get.getResponseBody();
> > > > > >                                 get.recycle();
> > > > > >                                 System.out.println(i + ", response:
" + code);
> > > > > >                         }
> > > > > >
> > > > > >                 } catch (Exception e) {
> > > > > >                         e.printStackTrace();
> > > > > >                 } finally {
> > > > > >                         get.releaseConnection();
> > > > > >
> > > > > >                 }
> > > > > >         }
> > > > > >
> > > > > > I checked the tutorial and I've made sure to read the response
body. I
> > > > > > call recycle() in the loop and yet it still uses a new connection
> > > > > > every time.
> > > > > >
> > > > > > All help very much appreciated.
> > > > > >
> > > > >
> > > > > Please see item 4 in the micro FAQ below
> > > > >
> > > > > http://tangentsoft.net/wskfaq/articles/debugging-tcp.html
> > > > >
> > > > > Oleg
> > > > >
> > > > >
> > > > > > Cheers,
> > > > > > Brad.
> > > > > >
> > > > > > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> > > > > > For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> > > > > For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
> > > > >
> > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> > > > For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
> > > >
> > > >
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org


Mime
View raw message