hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tomek Maciejewski <tomek.maciejew...@gmail.com>
Subject Re: BindException problem in multithreaded http client
Date Thu, 04 Sep 2008 13:33:25 GMT
Thanks Oleg, I supposed it could be a problem. I will change crawler 
architecture a little, to generate less connections simultaneously. So 
OS will have more time to release the sockets. I saw that it is possible 
to change the number of client ports in WinXP but I'm not sure if it 
would be a proper way to deal with this kind of problem.

Cheers,
Tomek

Oleg Kalnichevski wrote:
> On Wed, 2008-09-03 at 16:03 -0400, Tomek Maciejewski wrote:
>   
>> Hi Oleg,
>>
>> Thank for your reply. I have got the latest code from repository, but 
>> the problem didn't disappear. I saved the logs here: 
>> http://pentium.hopto.org/~thinred/files/output.zip
>> At the end of the crawler execution I started to get BindException. I 
>> would appreciate if you could see this logs, because actually I don't 
>> understand what is a relation between not reused connections and this 
>> bind exception. What is the nature of this problem, and why it tries to 
>> bind to 'address already in use'? Is there any way to solve this problem?
>>
>> Cheers,
>> Tomek
>>
>>
>>     
>
> It is all pretty simple. Please examine response messages in the log.
> Most of them (if not all) have "Connection: close" header, which means
> the server does not want the connection to be kept alive. As a result
> connections do not get reused. Every time a new request is executed, a
> new socket has to be open. Every time a new socket is open, a client
> port has to be allocated. Windows (per default) allocates the range from
> 1024 to 5000 for client ports. It usually takes a while before a used
> client port gets released back to the pool. If you execute 4000 requests
> in a tight loop you are pretty much bound to run out of client ports and
> end up with a bunch of BindExceptions
>
> Hope this helps
>
> Oleg
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>
>   


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org


Mime
View raw message