hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Kalnichevski <ol...@apache.org>
Subject Re: Cannot transmit a SWA response with a large attachment (HttpEntity can't chunk multipart)
Date Tue, 06 Jun 2017 18:30:21 GMT
On Mon, 2017-06-05 at 19:33 +0000, Dan Wlodarski wrote:
> Adding the ContentType parameter to the setEntity(...) call does
> reintroduce 
> the Content-Type line to the primary HTTP header. SoapUI now
> successfully 
> receives the entire SWA message. Thanks for all your help, Oleg!
> 
> Question: what was the design decision that led to the Content-Type
> header 
> ever being omitted from any HTTP message?
> 

What exactly do you mean? Some entity implementations automatically set
Content-Type when known but general purpose implementations such as
ByteArrayEntity or StringEntity expect Content-Type be set by the
caller.

Oleg

> Again, thanks,
> Dan C. W.
> 
> -----Original Message-----
> From: Oleg Kalnichevski [mailto:olegk@apache.org]
> Sent: Monday, June 5, 2017 12:55
> To: HttpClient User Discussion <httpclient-users@hc.apache.org>
> Subject: Re: Cannot transmit a SWA response with a large attachment 
> (HttpEntity can't chunk multipart)
> 
> On Mon, 2017-06-05 at 16:46 +0000, Dan Wlodarski wrote:
> > The source I ran was as provided in the updated Pastebin:
> > https://pastebin.com/2Hzdhpt3
> > 
> > How would I go about setting the content type of an HttpEntity
> > instance?
> 
> Something like that?
> 
> ---
> try (ByteArrayOutputStream out = new ByteArrayOutputStream()) {
>     responseEntity.writeTo(out);
>     response.setEntity(new ByteArrayEntity(out.toByteArray(), 
> ContentType.get(responseEntity)));
> }
> ---
> 
> Oleg
> 
> > The
> > interface doesn't include any mutators in its contract.
> > 
> > The getContentType() method of the HttpEntity instance built by
> > MultipartEntityBuilder does indeed return a string (ex. "Content-
> > Type:
> > multipart/form-data; boundary=rfNtJUy3v1e0XsLoZmxsYhXSU8k9LohLG"),
> > but the
> > Content-Type header doesn't get included in the response.
> > 
> > -----Original Message-----
> > From: Oleg Kalnichevski [mailto:olegk@apache.org]
> > Sent: Monday, June 5, 2017 10:58
> > To: HttpClient User Discussion <httpclient-users@hc.apache.org>
> > Subject: Re: Cannot transmit a SWA response with a large attachment
> > (HttpEntity can't chunk multipart)
> > 
> > On Mon, 2017-06-05 at 14:39 +0000, Dan Wlodarski wrote:
> > > Running the sample server code on Netbeans v8.1 with SoapUI
> > > v5.3.0
> > > as
> > 
> > ...
> > 
> > > 
> > > 
> > > Please note the missing Content-Type HTTP header.
> > > 
> > 
> > Did you set a Content-Type header on the response entity?
> > 
> > Oleg
> > 
> > 
> > > The Content-Type header is also missing from the response in my
> > > own
> > > curl
> > > v7.53.0 tests.
> > > 
> > > The additional testing and verification is appreciated.
> > > 
> > > -----Original Message-----
> > > From: Oleg Kalnichevski [mailto:olegk@apache.org]
> > > Sent: Monday, June 5, 2017 10:15
> > > To: HttpClient User Discussion <httpclient-users@hc.apache.org>
> > > Subject: Re: Cannot transmit a SWA response with a large
> > > attachment
> > > (HttpEntity can't chunk multipart)
> > > 
> > > On Mon, 2017-06-05 at 14:07 +0000, Dan Wlodarski wrote:
> > > > I've tried that solution previously. The result is a missing
> > > > initial HTTP header. The transmission begins at the MIME
> > > > multipart
> > > > border.
> > > > 
> > > 
> > > I see no evidence supporting this assertion.
> > > 
> > > Oleg
> > > 
> > > ---
> > > oleg@ok2c:~$ curl -X POST http://localhost:8080/ -v
> > > *   Trying ::1...
> > > * TCP_NODELAY set
> > > * Connected to localhost (::1) port 8080 (#0)
> > > > POST / HTTP/1.1
> > > > Host: localhost:8080
> > > > User-Agent: curl/7.52.1
> > > > Accept: */*
> > > > 
> > > 
> > > < HTTP/1.1 200 OK
> > > < Date: Mon, 05 Jun 2017 14:13:23 GMT < Server: Test/1.1 <
> > > Content-Length: 678 < Content-Type: multipart/form-data;
> > > boundary=PLjXq6xrj2ONaDdnyMPLBEnpkMqZA9Wmjw23
> > > <
> > > --PLjXq6xrj2ONaDdnyMPLBEnpkMqZA9Wmjw23
> > > Content-Disposition: form-data; name="SOAP Envelope"
> > > Content-Type: application/xml; charset=ISO-8859-1
> > > Content-Transfer-Encoding: 8bit
> > > 
> > > <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/
> > > en
> > > ve
> > > lope/">;;
> > >    <soapenv:Header/>
> > >    <soapenv:Body>
> > >       <AResponse>
> > >          <ResponseFile>/home/oleg/blah.txt</ResponseFile>
> > >       </AResponse>
> > >    </soapenv:Body>
> > > </soapenv:Envelope>
> > > --PLjXq6xrj2ONaDdnyMPLBEnpkMqZA9Wmjw23
> > > Content-Disposition: form-data; name="/home/oleg/blah.txt";
> > > filename="blah.txt"
> > > Content-Type: application/octet-stream
> > > Content-Transfer-Encoding: binary
> > > 
> > > blah
> > > 
> > > --PLjXq6xrj2ONaDdnyMPLBEnpkMqZA9Wmjw23--
> > > * Curl_http_done: called premature == 0
> > > * Connection #0 to host localhost left intact
> > > ---
> > > 
> > > > Pastebin source (https://pastebin.com/2Hzdhpt3) has been
> > > > updated
> > > > for further error replication.
> > > > 
> > > > -----Original Message-----
> > > > From: Oleg Kalnichevski [mailto:olegk@apache.org]
> > > > Sent: Monday, June 5, 2017 07:14
> > > > To: HttpClient User Discussion <httpclient-users@hc.apache.org>
> > > > Subject: Re: Cannot transmit a SWA response with a large
> > > > attachment (HttpEntity can't chunk multipart)
> > > > 
> > > > On Sun, 2017-06-04 at 15:03 +0000, Dan Wlodarski wrote:
> > > > > To whom it may concern:
> > > > > 
> > > > > Issue: HttpClient API does not support chunking multipart
> > > > > HttpEntities.
> > > > > Clients making SOAP requests to a mock SOAP with attachments
> > > > > (SWA)
> > > > > server are failing (specifically, Apache-based Java clients
> > > > > are
> > > > > throwing
> > > > > org.apache.http.NoHttpResponseExceptions) when the attachment
> > > > > in
> > > > > the response is large (>2 MiB). Small attachments are
> > > > > transmitted without throwing any exceptions. HTTP Content-
> > > > > Length
> > > > > header is correct.
> > > > > 
> > > > > Use case: Multithreaded SOAP server capable of transmitting
> > > > > SWA
> > > > > responses of arbitrary content length
> > > > > 
> > > > > Libraries: HttpCore v4.4.6 + HttpMime v4.5.3
> > > > > 
> > > > > Example server source for error replication is available at:
> > > > > https://pastebin.com/2Hzdhpt3
> > > > > 
> > > > > Error replication:
> > > > > 1. Build the above source with HttpCore v4.4.6 and HttpMime
> > > > > v4.5.3
> > > > > libraries (HttpMime is part of the HttpClient project).
> > > > > 2. Run the program with a sufficiently large (>2 MiB) binary
> > > > > file named "random.png.gz" with correct pathing and
> > > > > permissions
> > > > > (a readable directory sibling of the executable).
> > > > > 3. Send the server an arbitrary HTTP POST request via some
> > > > > third-
> > > > > party client. Note the failure to receive the large server-
> > > > > generated multipart result.
> > > > > 
> > > > > NB: SoapUI can be leveraged as an Apache-based Java client
> > > > > with
> > > > > precision logging.
> > > > > 
> > > > > Please advise.
> > > > > 
> > > > 
> > > > Dan,
> > > > 
> > > > You are mixing up non-blocking i/o transport and a blocking i/o
> > > > entity implementation. This is generally a bad idea as it
> > > > cannot
> > > > be done efficiently without full or partial content buffering
> > > > in
> > > > memory.
> > > > 
> > > > If do not mind buffering the entire entity in memory, you can
> > > > replace
> > > > ---
> > > > response.setEntity(responseEntity);
> > > > ---
> > > > 
> > > > with
> > > > 
> > > > ---
> > > > final ByteArrayOutputStream out = new ByteArrayOutputStream();
> > > > responseEntity.writeTo(out); response.setEntity(new
> > > > ByteArrayEntity(out.toByteArray()));
> > > > ---
> > > > 
> > > > and your code will work just fine.
> > > > 
> > > > Oleg
> > > > 
> > > > 
> > > > > Thanks,
> > > > > 
> > > > > Dan C. Wlodarski
> > > > > The Design Knowledge Company
> > > > > 3100 Presidential Drive
> > > > > Suite 103
> > > > > Fairborn, Ohio 45324
> > > > > 
> > > > > Phone: 937-427-4276 x175
> > > > > Fax: 937-427-1242
> > > > > dwlodarski@tdkc.com
> > > > > www.tdkc.com
> > > > > 
> > > > > P.S. This issue was originally submitted on 30 May, but did
> > > > > not
> > > > > appear in the archives. Assumed dropped.
> > > > > 
> > > > > 
> > > > > -------------------------------------------------------------
> > > > > ----
> > > > > ----
> > > > > To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apach
> > > > > e.
> > > > > or
> > > > > g
> > > > > For additional commands, e-mail: httpclient-users-help@hc.apa
> > > > > ch
> > > > > e.
> > > > > or
> > > > > g
> > > > > 
> > > > 
> > > > -------------------------------------------------------------
> > > > ----
> > > > ----
> > > > To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.
> > > > or
> > > > g
> > > > For additional commands, e-mail: httpclient-users-help@hc.apach
> > > > e.
> > > > or
> > > > g
> > > > 
> > > 
> > > -----------------------------------------------------------------
> > > ----
> > > To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.or
> > > g
> > > For additional commands, e-mail: httpclient-users-help@hc.apache.
> > > or
> > > g
> > > 
> > 
> > -----------------------------------------------------------------
> > ----
> > To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> > For additional commands, e-mail: httpclient-users-help@hc.apache.or
> > g
> > 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org


Mime
View raw message