hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Kalnichevski <ol...@apache.org>
Subject Re: HttpAsyncClient - Connection forcibly closed by remote host
Date Thu, 23 Feb 2012 19:20:50 GMT
On Thu, 2012-02-23 at 16:22 +0100, Alexander Hjalmarsson wrote:
> I've created a simple crawling framework with the help from HttpAsyncClient
> and sometimes I get the exception that an existing connection was forcibly
> closed by the remote host. I'm doing the requests rather polite (one
> request every 15 seconds) so I'm not really overwhelming the server. What
> could be the cause for this? I'd also like to know how/where I can catch
> this exception myself, so I don't continue to send requests to a server
> that closes the connections.
> 
> 2012-feb-23 16:04:59
> org.apache.http.impl.nio.client.LoggingClientProtocolHandler exception
> SEVERE: http-outgoing-259 [ACTIVE] I/O error: An existing connection was
> forcibly closed by the remote host
> java.io.IOException: An existing connection was forcibly closed by the
> remote host

Things like that can happen. HTTP servers can and do drop connections
for all sorts of reasons. Your crawler should be prepared to deal with
such situations. I also think there is absolutely no reason to blacklist
the server just because of one failed request. The server should respond
with status 503 (Service Unavailable) if it is unable to process the
request due to overload or a similar condition.

Oleg


>     at sun.nio.ch.SocketDispatcher.read0(Native Method)
>     at sun.nio.ch.SocketDispatcher.read(Unknown Source)
>     at sun.nio.ch.IOUtil.readIntoNativeBuffer(Unknown Source)
>     at sun.nio.ch.IOUtil.read(Unknown Source)
>     at sun.nio.ch.SocketChannelImpl.read(Unknown Source)
>     at
> org.apache.http.impl.nio.reactor.SessionInputBufferImpl.fill(SessionInputBufferImpl.java:95)
>     at
> org.apache.http.impl.nio.codecs.AbstractMessageParser.fillBuffer(AbstractMessageParser.java:115)
>     at
> org.apache.http.impl.nio.conn.DefaultAsyncClientConnection$LoggingNHttpMessageParser.fillBuffer(DefaultAsyncClientConnection.java:164)
>     at
> org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:153)
>     at
> org.apache.http.impl.nio.DefaultClientIODispatch.onInputReady(DefaultClientIODispatch.java:118)
>     at
> org.apache.http.impl.nio.DefaultClientIODispatch.onInputReady(DefaultClientIODispatch.java:49)
>     at
> org.apache.http.impl.nio.AbstractIODispatch.inputReady(AbstractIODispatch.java:113)
>     at
> org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:160)
>     at
> org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:342)
>     at
> org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:320)
>     at
> org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:280)
>     at
> org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:106)
>     at
> org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:599)
>     at java.lang.Thread.run(Unknown Source)
> 
> The requests I make are just basic GET-requests, executed like following:
> 
> httpclient.execute(request, new FutureCallback<HttpResponse>() {
>                         public void failed(Exception arg0) {
>                             //Do nothing
>                         }
>                         public void completed(HttpResponse arg0) {
>                             EntityUtils.consume(arg0.getEntity());
>                         }
> 
>                         public void cancelled() {
>                             //Do nothing
>                         }
> });



---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org


Mime
View raw message