hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Kalnichevski <ol...@apache.org>
Subject Re: BindException problem in multithreaded http client
Date Wed, 03 Sep 2008 12:31:32 GMT
Tomek,

I improved some of the DEBUG log statements and reduced the amount of
noise in the logs. Feel free to get the latest code snapshot from SVN
trunk and retest with your application.

Cheers

Oleg

On Wed, 2008-09-03 at 12:27 +0200, Oleg Kalnichevski wrote:
> On Tue, 2008-09-02 at 17:57 -0400, Tomek Maciejewski wrote:
> > Thanks Oleg, now it works. Earlier I used System.setProperty to enable 
> > logging but I get only WARN level logging, I don't know if the 
> > properties are case-sensitive because I set the same properties. Only 
> > difference was that I wrote "debug" instead of "debug".
> > Generally, with the debug enabled I don't get the exception. Maybe it is 
> > because, the crawler runs a little slower because of logging operation 
> > and saving them to output?
> 
> Can be.
> 
> > In the logs I found that the connection is created 3715 times and in the 
> > client I have the limit of 100 connections. Generally I often see 
> > something like that:
> > 
> > [DEBUG] ThreadSafeClientConnManager - Released connection open but not 
> > marked reusable.
> > [DEBUG] DefaultClientConnection - Connection shut down
> > [DEBUG] ConnPoolByRoute - Freeing connection 
> > [HttpRoute[{}->http://www.mrw.interscience.wiley.com]][null]
> > [DEBUG] ConnPoolByRoute - Creating new connection 
> > [HttpRoute[{}->http://www.mrw.interscience.wiley.com]]
> > [DEBUG] ThreadSafeClientConnManager - 
> > ThreadSafeClientConnManager.getConnection: 
> > HttpRoute[{}->http://www.mrw.interscience.wiley.com], timeout = 0
> > 
> > So the connection is not marked as reusable? Could it be a reason of my 
> > BindException problems?
> 
> Yes, this is precisely the reason. I cannot say why the connection got
> marked as non-reusable in the first place without seeing the complete
> context log.
> 
> Oleg
> 
> > I execute connections very simultaneously, right now there may be about 
> > 600 threads in executors and every of them invokes 'execute method' on 
> > httpclient object.
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org


Mime
View raw message