Mailing-List: contact httpclient-user-help@jakarta.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "HttpClient User Discussion" <httpclient-user@jakarta.apache.org>
Received-SPF: neutral (hermes.apache.org: local policy)
Date: Tue, 31 May 2005 09:41:09 +0200
From: Oleg Kalnichevski <olegk@apache.org>
To: HttpClient User Discussion <httpclient-user@jakarta.apache.org>,
  The Book Grove <bookgrove@gmail.com>
Subject: Re: Thousands of Sockets in LISTENING state
Message-ID: <20050531074109.GA20109@uml24.umlhosting.ch>
Mail-Followup-To: HttpClient User Discussion
 <httpclient-user@jakarta.apache.org>,
	The Book Grove <bookgrove@gmail.com>
References: <62483ebb050529170511a10f61@mail.gmail.com>
 <20050530085147.GA19137@uml24.umlhosting.ch>
 <62483ebb0505300851358e8b03@mail.gmail.com>
 <20050530160935.GA19571@uml24.umlhosting.ch>
 <62483ebb050530202131199704@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
In-Reply-To: <62483ebb050530202131199704@mail.gmail.com>
User-Agent: Mutt/1.3.28i

On Mon, May 30, 2005 at 10:21:53PM -0500, The Book Grove wrote:
> Oleg,
> 
> Wow! You were right on the money with suggesting to revert back to
> 1.4.2 JRE. I looped the code 100,000 times and it re-uses just one
> measly socket the entire time.

This is not unexpected but quite troubling nevertheless. I'll try to run
some tests with HttpClient in JRE 1.5 sometime this weekend

I should have posted my problem on the
> mailing lists sooner to save so much time and frustration. Can you
> please post the link to JavaDocs for HttpClient 3.0 just to be certain
> that I am looking in the right spot for getParams() method and
> everything else. Again, excellent diagnostics. I will write back again
> if another problem arrises.

http://jakarta.apache.org/commons/httpclient/3.0/apidocs/index.html

You may also want to check out the following HttpClient tutorials

http://jakarta.apache.org/commons/httpclient/3.0/tutorial.html
http://jakarta.apache.org/commons/httpclient/3.0/preference-api.html

Cheers,

Oleg

> 
> Note: Switching from JRE 5.0 to JRE 1.4.2_08 took my whole Resin
> Servlet Server down for about 2 hours until I figured out that I had
> to delete all of the Servlet .class files so that Resin could
> recompile the .java files with the new 1.4.2_08 environment. Hope this
> helps.
> 
> -Bailey-
> 
> On 5/30/05, Oleg Kalnichevski <olegk@apache.org> wrote:
> > On Mon, May 30, 2005 at 10:51:15AM -0500, The Book Grove wrote:
> > > Oleg,
> > >
> > > Requested information is as follows:
> > >
> > > (1) HttpClient 3.0 rc2
> > > (2) JRE 5.0
> > > (3) Windows XP Professional
> > 
> > Eeeehhhhh...
> > 
> > 
> > > (4) SimpleHttpConnectionManager
> > > (5) N/A
> > >
> > > Using the SimpleConnectionManager, is there a way to configure it so
> > > that it will reuse a single socket per request, given that
> > 
> > This is precisely what SimpleHttpConnectionManager does. It uses a
> > single connection and keeps it alive as long as it can be reused.
> > 
> > http://jakarta.apache.org/commons/httpclient/3.0/xref/org/apache/commons/httpclient/SimpleHttpConnectionManager.html
> > 
> > 
> > > HttpMethod#releaseConnection, or force it to be dropped/closed or at
> > > least in CLOSE_WAIT state (i've forgotten the exact phrase for sockets
> > > in this state).
> > 
> > Feel free to extend SimpleConnectionManager, override the
> > releaseConnection method and close the connection prior to calling
> > super#releaseConnection. However, I beleive the problem has nothing to
> > do with the SimpleConnectionManager, rather with your OS/JRE combo
> > 
> > >
> > > Note: I've messed around with the MultiThreadedHttpConnectionManager
> > > and didn't know exactly all of the configurations that could be set
> > > with it, or if I even needed to be using it for my problems. Also,
> > > I've seen in previous mailing archives code like
> > > "HttpConnectionManager.getParams()". I've been looking in the JavaDocs
> > > and at Jakarta Common's HttpClient site to learn how to use these
> > > tools, and haven't found yet where getParams() comes from.
> > 
> > Make sure you are looking at the javadocs for HttpClient 3.0
> > 
> > Try running your test app in Java 1.4.2 and see if you observe similar
> > problem(s)
> > 
> > Oleg
> > 
> > >
> > > Thanks in advance for your assistance.
> > >
> > > -Bailey-
> > >
> > > On 5/30/05, Oleg Kalnichevski <olegk@apache.org> wrote:
> > > > Bailey,
> > > >
> > > > First of all, HttpMethod#releaseConnection does not necessarily close
> > > > the underlying socket. If the connection can be reused, it will be kept
> > > > alive indefinitely until closed either on the server or on the client
> > > > side.
> > > >
> > > > Depending on the connection manager used and its configuration it
> > > > is entirely possible that thousands of connections will end up in the
> > > > LISTENING state, if no provisions have been made to evict connections
> > > > from the connection pool.
> > > >
> > > > (1) which version of HttpClient are you using?
> > > > (2) which JRE version?
> > > > (3) which platform/OS?
> > > > (4) which connection manager?
> > > > (5) if you are using a pooling connection manager such as
> > > > MultiThreadedHttpConnectionManager, what are connection pool settings?
> > > >
> > > > Oleg
> > > >
> > > > On Sun, May 29, 2005 at 07:05:28PM -0500, The Book Grove wrote:
> > > > > I've written many programs and methods using HttpClient, and have
> > > > > noticed that if I leave them running long enough, I will eventually
> > > > > get JVM BindExceptions. Then I use "netstat -a" to look at the
> > > > > thousands of sockets in the LISTENING state. I wrote a short little
> > > > > program to produce the same socket problems. Below is a summary:
> > > > >
> > > > > HttpClient httpClient = new HttpClient();
> > > > > while(loop this 100,000 times)
> > > > > {
> > > > > Protocol protocol = new Protocol("https", new
> > > > > EasySSLProtocolSocketFactory(), 443);
> > > > > httpClient.getHostConfiguration().setHost("www.foobar.com", 80, "http");
> > > > > GetMethod getMethod = new GetMethod("/foobar.html");
> > > > > httpClient.executeMethod(getMethod);
> > > > > //load response into linked list with BufferedReader, then close
> > > > > reader when done
> > > > > httpClient.releaseConnection();
> > > > > }
> > > > >
> > > > > Normally the releaseConnection() should clean up all resources,
> > > > > correct? I don't think that this would be a problem if I could get the
> > > > > sockets in a TIME_WAIT state (or whatever it's called). But since it
> > > > > is in LISTENING state, nothing is done to regain them, not by Garbage
> > > > > Collector and not by Operating System.
> > > > >
> > > > > Note: This is when running the test program in DOS using .bat files to
> > > > > set classpaths and run the .class file. When using NetBeans, the
> > > > > sockets are in "ESTABLISHED" state, but eventually I get the same
> > > > > complaint about the sockets.
> > > > >
> > > > > Thanks in advance for your time and consideration.
> > > > >
> > > > > -Bailey-
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> > > > > For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
> > > > >
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
> > >
> >
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org