hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jon Moore <j...@apache.org>
Subject Re: How to stream a file via PUT to multiple HttpClients
Date Wed, 20 Jul 2011 13:37:04 GMT
Hi Sachin,

In this case, I would suggest running the .execute() calls in separate
threads, or perhaps using the asynchronous HttpClient:
http://hc.apache.org/httpcomponents-asyncclient-dev/index.html

I think those are your only options given the existing APIs for HttpClient.

Jon

On Wed, Jul 20, 2011 at 9:20 AM, Sachin Shetty <sshetty@egnyte.com> wrote:
> But since I would execute client.execute(put1) and client.execute(put2)
> serially in a single thread, ForkedInputStream will end up buffering almost
> the entire file for put2 which I want to avoid at any cost.
>
> In short, I want to spawn two puts and write to them in blocks one after
> another.
>
> -----Original Message-----
> From: Jon Moore [mailto:jonm@apache.org]
> Sent: Wednesday, July 20, 2011 6:42 PM
> To: HttpClient User Discussion
> Subject: Re: How to stream a file via PUT to multiple HttpClients
>
> Hi Sachin,
>
> I think what you want to do is wrap your FileInputStream in another
> object, let's call it a ForkedInputStream:
>
> FileInputStream fis = ...;
> ForkedInputStream forked = new ForkedInputStream(fis);
> HttpPut put1 = new HttpPut(...);
> HttpPut put2 = new HttpPut(...);
> put1.setEntity(new InputStreamEntity(forked.getLeft()));
> put2.setEntity(new InputStreamEntity(forked.getRight()));
>
> etc.
>
> Then your ForkedInputStream takes care of reading from the underlying
> File (perhaps on demand) and keeping the multiple writers in as much
> lockstep or with as much buffering as desired.
>
> Jon
>
> On Wed, Jul 20, 2011 at 6:02 AM, Sachin Shetty <sshetty@egnyte.com> wrote:
>> Hi,
>>
>>
>>
>> We have a unique case where we want to put a file via HTTP PUT to more
> than
>> one http servers  but we want to read the file only once which means we
> want
>> to read a block of data from the file and write  to two entities
>> simultaneously. Streaming the PUT and reading the file only once without
>> having to hold it in memory are the two things we cannot avoid.
>>
>>
>>
>> These things are pretty easy when we have a direct outputstream to write
> to,
>> but I am not able to figure out how this can be achieved using the
> entities.
>>
>>
>>
>> basically we want an httpclient equivalent of:
>>
>>
>>
>> OutputStream stream1 = ...
>>
>> OutputStream stream2 = ...
>>
>>
>>
>> while (input.read(byte[1024]) {
>>
>>   stream1.write(bytes);
>>
>>   stream2.write(bytes);
>>
>> }
>>
>>
>>
>> Please let me know if there is a way out?
>>
>>
>>
>> Thanks
>>
>> Sachin
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org


Mime
View raw message