Mailing-List: contact commons-httpclient-dev-help@jakarta.apache.org;
 run by ezmlm
Message-ID: <3F2E8A53.2030307@centrum.cz>
Date: Mon, 04 Aug 2003 18:31:15 +0200
From: Leo Galambos <leog@centrum.cz>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US;
 rv:1.4) Gecko/20030624
MIME-Version: 1.0
To: commons-httpclient-dev@jakarta.apache.org
Subject: Connection break
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit

Hi.

I write a robot for a search engine. The robot must harvest all files 
which are shorter than a few kilobytes (let's say 100kB) - longer  files 
are not important, because they are often archives or long sheets about 
nothing.

I cannot find a robust style in which I could drop a connection (GET 
over HTTP/1.0 and HTTP/1.1) when the incoming data stream exceeds the 
upper limit. I do it by closing the input stream, which is constructed 
by getResponseAsStream, followed by releaseConnection. Is it OK?

My second point is related to "retrying" you have in your docs 
(http://jakarta.apache.org/commons/httpclient/tutorial.html - catch 
block of HttpRecovableException). When I do something like this, I found 
out that I had to call method.recycle() in the catch block, or the 
connection was not reinitialized and everything fails. Could you 
enlighten me on this? Is it a bug in the guide? (I have tried it on 2.0-b1).

And my last point - when I run the robot under stress conditions, some 
connections seem to be frozen, although I use setConnectionTimeout. Is 
it a known issue? How should I debug it so that you can get a valuable 
log? It happens after 1-2 hours of run, so the log could have a few gigas...

Thank you

-g-