hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jaikit Savla <jaikit.sa...@yahoo.com.INVALID>
Subject Re: Using HTTPClient for mulithreaded HTTP Get download
Date Tue, 14 Oct 2014 22:21:15 GMT
Yes. As Oleg mentioned, critical thing is to have1. Singleton instance of HTTPClient which
will be used for making all requests. This will make sure you reuse the same connections (
connection pooling)
2. While configuring your Httpclient, use Pooling Httpclient connection manager. 
Jaikit 

     On Monday, October 13, 2014 7:23 AM, Arnold Maderthaner <arnold@maderthaner.me>
wrote:
   

 Hi !

So I can do something similar like here
(http://hc.apache.org/httpcomponents-client-4.3.x/httpclient/examples/org/apache/http/examples/client/ClientMultiThreadedExecution.java)
using ExecutorService than "normal" threads (just need to use the
PoolingHttpClientConnectionManager) ?

yours

Arnold

On Mon, Oct 13, 2014 at 3:50 PM, Oleg Kalnichevski <olegk@apache.org> wrote:
> On Mon, 2014-10-13 at 11:59 +0200, Arnold Maderthaner wrote:
>> Hi !
>>
>> I'm trying to write a simple application that does a multi threaded
>> HTTP Get download from a server which needs to download approx. 2000
>> files.
>> I have a list of URLs that I want to fetch and want the binary content
>> to be stored either on the filesystem or in memory.
>> Depending on configuration I want to use 10 threads to download this
>> from the server.
>>
>> What is the best method to implement this ? Should I use an
>> ExecutorService with a fixed thread pool and create an own HTTPClient
>> for each thread or is there some better (less expensive) approach ?
>>
>
> You should be using multiple worker threads (by means of ExecutorService
> or otherwise) but make worker threads share the same instance of
> HttpClient.
>
> Oleg
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org



   
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message