hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jon Brisbin <...@jbrisbin.com>
Subject Re: Using async HTTP client to upload 1GB entity
Date Tue, 09 Aug 2011 19:34:32 GMT
I did look at those but I wasn't able to divine from them what I need to adapt to my situation.
I'm not starting with a File. I'm not use a FileChannel, either. I have simply a chunk of
data that needs to be streamed to a PUT endpoint. I need to call ContentEncoder.write() each
time I get a chunk. My question is basically, if I do this, does the ByteBuffer get sent down
the wire to the server so I can reclaim that buffer and call write() again with a different
chunk of data? 


Jon Brisbin 

----- Original Message -----

> From: "Oleg Kalnichevski" <olegk@apache.org>
> To: "HttpComponents Project" <dev@hc.apache.org>
> Sent: Tuesday, August 9, 2011 2:28:48 PM
> Subject: Re: Using async HTTP client to upload 1GB entity

> On Tue, 2011-08-09 at 13:26 -0500, Jon Brisbin wrote:
> > I can't figure out how I need to adapt the various
> > Producer/Consumer components to do a large file upload . I do not
> > have the file on the local filesystem and I obviously don't want
> > to buffer the entire file in memory if I want to do more than a
> > single upload at a time. :)
> >
> > I'm looking into creating an async Producer to do this, but I'm
> > wondering how it works underneath. If I call write() on the
> > ContentEncoder with a ByteBuffer, does it flush the bytes down the
> > channel so that they aren't in memory any longer? I don't want to
> > accumulate this stuff until the end of the upload then send
> > everything at once. I want to stream it to the server as I get it.
> >
> >
> > Thanks!
> >
> > Jon Brisbin
> > http//jbrisbin.com
> >
> >

> Jon,

> Have you looked at the ZeroCopyPost / ZeroCopyPut producers? They are
> precisely meant for efficient (zero copy) file transfers.

> Oleg

> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> For additional commands, e-mail: dev-help@hc.apache.org

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message