hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Keyes <j...@integralsource.com>
Subject Re: streaming request body
Date Tue, 24 Feb 2004 14:19:09 GMT
> For (a), Oleg's response is correct. You might easily be confused, in  
> the sense that HttpClient's API inverts the control. It is not that  
> you write to an "OutputStream" to send your data, it is that you  
> provide HttpClient with an "InputStream", and it reads that stream and  
> sends the data. HttpClient is designed to accomodate your concern, and  
> if your configuration is correct (as per the examples), it will not  
> buffer the entire contents of your InputStream, but rather read it and  
> send it in small chunks. As another post points you, you may still  
> have to buffer what you're sending to *disk*, but not to memory.

So you think buffering all requests to disk to support streaming is an  
acceptable solution?  If I am dealing with XXX,000 of requests that  
sure as hell would suck with all the disk I/O going on.  Does this not  
suggest that there is a problem with the architecture?

> As for (b), this is again under your control via  
> HttpMethod.getResponseBodyAsStream(). As with (a), you can also invoke  
> HttpClient such that it does cache the entire contents  
> (HttpMethod.getResponseBodyAsString() ).
>
> In both cases, it is possible to get the behavior that you desire.

Not it is not.  Again think of XXX,000 of requests.

> Connection pooling is only part of the concern. HttpClient supports  
> HTTP 1.1 persistent connections. It doesn't expose the underlying  
> socket's InputStream and OutputStream. If it did, it cound not ensure  
> that persistent connections work properly.

I still don't see the problem.  The OutputStream and InputStream can be  
wrapped so there is no loss of control.  Why do you think control would  
be lost?

-John K

>
> -Eric.
>
> John Keyes wrote:
>
>> Guys,
>>
>> A colleague pointed out to me that this does not in fact resolve the  
>> situation. The solutions pointed out allow me to read the attachment  
>> as a stream. The contents are still held in memory prior to writing  
>> it on the wire. To fully support this you would need access to the  
>> OutputStream.
>>
>> If we could pass a HttpClient to the HttpMethod then we could get  
>> access to the output stream via the getRequestOutputStream method.
>>
>> I don't understand the connection pooling argument. I thought it  
>> should be a user preference whether to have connection pooling.
>>
>> Any ideas on this?
>> -John K
>>
>> On 23 Feb 2004, at 13:02, Kalnichevski, Oleg wrote:
>>
>>>
>>> John,
>>>
>>> HttpClient's entity enclosing methods (POST, PUT) do support content  
>>> streaming when (1) the content length does not need to be  
>>> automatically calculated or (2) chunk-encoding is used
>>>
>>> Please refer to the following sample applications for details
>>>
>>> Unbuffered post:
>>>
>>> http://cvs.apache.org/viewcvs.cgi/*checkout*/jakarta-commons/  
>>> httpclient/src/examples/UnbufferedPost.java?content-  
>>> type=text%2Fplain&rev=1.2.2.1
>>>
>>> Chunk-encoded post:
>>>
>>> http://cvs.apache.org/viewcvs.cgi/*checkout*/jakarta-commons/  
>>> httpclient/src/examples/ChunkEncodedPost.java?content-  
>>> type=text%2Fplain&rev=1.4.2.1
>>>
>>> Hope this helps
>>>
>>> Oleg
>>>
>>>
>>> -----Original Message-----
>>> From: John Keyes [mailto:john@integralsource.com]
>>> Sent: Monday, February 23, 2004 13:54
>>> To: commons-httpclient-dev@jakarta.apache.org
>>> Subject: streaming request body
>>>
>>>
>>> Hi,
>>>
>>> I notice you have separated out the functions of the connection and  
>>> the
>>> content creation. So the code must be something like
>>>
>>> HttpClient client = new HttpClient( url );
>>> HttpMethod method = new GetMethod();
>>> method.setRequestHeader( ... ); ...
>>> method.setRequestBody( ... );
>>> client.execute( method );
>>>
>>> If I want to send a large attachment and I don't want it all to be in
>>> memory then I can't do it. The issue is that you have to write your
>>> data to the HttpMethod. The HttpMethod doesn't know where to then  
>>> write
>>> this data until you call execute and pass the client which has the
>>> connection to write to. So there isn't really a way around this  
>>> because
>>> of the separation of the connection from the HttpMethod.
>>>
>>> So my question is, is there a way to stream the request body rather
>>> than having to store the request in memory prior to writing it on the
>>> wire.
>>>
>>> Thanks,
>>> -John K
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail:  
>>> commons-httpclient-dev-unsubscribe@jakarta.apache.org
>>> For additional commands, e-mail:  
>>> commons-httpclient-dev-help@jakarta.apache.org
>>>
>>>
>>> ********************************************************************* 
>>> ** ****************************
>>> The information in this email is confidential and may be legally  
>>> privileged. Access to this email by anyone other than the intended  
>>> addressee is unauthorized. If you are not the intended recipient of  
>>> this message, any review, disclosure, copying, distribution,  
>>> retention, or any action taken or omitted to be taken in reliance on  
>>> it is prohibited and may be unlawful. If you are not the intended  
>>> recipient, please reply to or forward a copy of this message to the  
>>> sender and delete the message, any attachments, and any copies  
>>> thereof from your system.
>>> ********************************************************************* 
>>> ** ****************************
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail:  
>>> commons-httpclient-dev-unsubscribe@jakarta.apache.org
>>> For additional commands, e-mail:  
>>> commons-httpclient-dev-help@jakarta.apache.org
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:  
>> commons-httpclient-dev-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail:  
>> commons-httpclient-dev-help@jakarta.apache.org
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:  
> commons-httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:  
> commons-httpclient-dev-help@jakarta.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: commons-httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-httpclient-dev-help@jakarta.apache.org


Mime
View raw message