hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Morris, Chris" <Chris.Mor...@epsilon.com>
Subject RE: HTTP Response contains headers last
Date Thu, 27 Sep 2007 15:48:59 GMT
Thanks Roland.  

I tried treating them as footers to no avail.  I still don't get access
to the actual body of the response.

Here's the wire log.  As you can see from below, the "data" portion of
the response (which should be considered the body) is being skipped over
to get to the headers.  The problem is that attempting to grab the data
as headers, footers or body is not working.  It's like the method is
throwing those lines away (but the wire output thinks they are header
rows).

[INFO] DreamAPIRequest - -Extracting activity for campaign: test
mailing: test01
[DEBUG] HttpClient - -Java version: 1.5.0_10
[DEBUG] HttpClient - -Java vendor: Sun Microsystems Inc.
[DEBUG] HttpClient - -Java class path: C:\Documents and
Settings\cmorris\workspace\AMI_2.0_Local\target;C:\Documents and
Settings\cmorris\workspace\AMI_2.0_Local\target\classes;C:\Documents and
Settings\cmorris\.m2\repository\commons-codec\commons-codec\1.2\commons-
codec-1.2.jar;C:\Documents and
Settings\cmorris\.m2\repository\commons-httpclient\commons-httpclient\3.
0.1\commons-httpclient-3.0.1.jar;C:\Documents and
Settings\cmorris\.m2\repository\commons-io\commons-io\1.3.1\commons-io-1
.3.1.jar;C:\Documents and
Settings\cmorris\.m2\repository\commons-logging\commons-logging\1.0.3\co
mmons-logging-1.0.3.jar;C:\Documents and
Settings\cmorris\.m2\repository\commons-vfs\commons-vfs\1.0\commons-vfs-
1.0.jar;C:\Documents and
Settings\cmorris\.m2\repository\jdom\jdom\1.0\jdom-1.0.jar;C:\Documents
and
Settings\cmorris\.m2\repository\junit\junit\3.8.1\junit-3.8.1.jar;C:\Doc
uments and
Settings\cmorris\workspace\AMI_2.0_Local\jar\commons-httpclient-3.0.1.ja
r;C:\Documents and
Settings\cmorris\workspace\AMI_2.0_Local\jar\commons-logging-1.0.3.jar;C
:\Documents and
Settings\cmorris\workspace\AMI_2.0_Local\jar\commons-vfs-20070719.jar;C:
\Documents and
Settings\cmorris\workspace\AMI_2.0_Local\jar\jdom-1.0.jar;C:\Documents
and Settings\cmorris\workspace\AMI_2.0_Local\jar\xerces.jar;C:\Documents
and
Settings\cmorris\workspace\AMI_2.0_Local\jar\commons-codec-1.2.jar;C:\Do
cuments and
Settings\cmorris\workspace\AMI_2.0_Local\jar\commons-net-1.4.1.jar;C:\Do
cuments and Settings\cmorris\workspace\AMI_2.0_Local\jar\jsch-0.1.32.jar
[DEBUG] HttpClient - -Operating system name: Windows XP
[DEBUG] HttpClient - -Operating system architecture: x86
[DEBUG] HttpClient - -Operating system version: 5.1
[DEBUG] HttpClient - -SUN 1.5: SUN (DSA key/parameter generation; DSA
signing; SHA-1, MD5 digests; SecureRandom; X.509 certificates; JKS
keystore; PKIX CertPathValidator; PKIX CertPathBuilder; LDAP, Collection
CertStores)
[DEBUG] HttpClient - -SunRsaSign 1.5: Sun RSA signature provider
[DEBUG] HttpClient - -SunJSSE 1.5: Sun JSSE provider(PKCS12, SunX509
key/trust factories, SSLv3, TLSv1)
[DEBUG] HttpClient - -SunJCE 1.5: SunJCE Provider (implements RSA, DES,
Triple DES, AES, Blowfish, ARCFOUR, RC2, PBE, Diffie-Hellman, HMAC)
[DEBUG] HttpClient - -SunJGSS 1.0: Sun (Kerberos v5)
[DEBUG] HttpClient - -SunSASL 1.5: Sun SASL provider(implements client
mechanisms for: DIGEST-MD5, GSSAPI, EXTERNAL, PLAIN, CRAM-MD5; server
mechanisms for: DIGEST-MD5, GSSAPI, CRAM-MD5)
[DEBUG] DefaultHttpParams - -Set parameter http.useragent = Jakarta
Commons-HttpClient/3.0.1
[DEBUG] DefaultHttpParams - -Set parameter http.protocol.version =
HTTP/1.1
[DEBUG] DefaultHttpParams - -Set parameter http.connection-manager.class
= class org.apache.commons.httpclient.SimpleHttpConnectionManager
[DEBUG] DefaultHttpParams - -Set parameter http.protocol.cookie-policy =
rfc2109
[DEBUG] DefaultHttpParams - -Set parameter http.protocol.element-charset
= US-ASCII
[DEBUG] DefaultHttpParams - -Set parameter http.protocol.content-charset
= ISO-8859-1
[DEBUG] DefaultHttpParams - -Set parameter http.method.retry-handler =
org.apache.commons.httpclient.DefaultHttpMethodRetryHandler@1198891
[DEBUG] DefaultHttpParams - -Set parameter http.dateparser.patterns =
[EEE, dd MMM yyyy HH:mm:ss zzz, EEEE, dd-MMM-yy HH:mm:ss zzz, EEE MMM d
HH:mm:ss yyyy, EEE, dd-MMM-yyyy HH:mm:ss z, EEE, dd-MMM-yyyy HH-mm-ss z,
EEE, dd MMM yy HH:mm:ss z, EEE dd-MMM-yyyy HH:mm:ss z, EEE dd MMM yyyy
HH:mm:ss z, EEE dd-MMM-yyyy HH-mm-ss z, EEE dd-MMM-yy HH:mm:ss z, EEE dd
MMM yy HH:mm:ss z, EEE,dd-MMM-yy HH:mm:ss z, EEE,dd-MMM-yyyy HH:mm:ss z,
EEE, dd-MM-yyyy HH:mm:ss z]
[INFO] HttpAPIRequest - -postRequest posting to:
http://dweb.bfi0.com:9000/api/ActivityDownload.mpl
[INFO] HttpAPIRequest - -HTTP Post contains the following parameters:
[INFO] HttpAPIRequest - -parameter: headers_first = 0
[INFO] HttpAPIRequest - -parameter: client_name = xxxxxxxxxxxx
[INFO] HttpAPIRequest - -parameter: client_token = xxxxxxxxxxxxx
[INFO] HttpAPIRequest - -parameter: output_field_delimiter = ||
[INFO] HttpAPIRequest - -parameter: campaign_name = test
[INFO] HttpAPIRequest - -parameter: mailing_name = test01
[INFO] HttpAPIRequest - -parameter: start_date = 20070901
[INFO] HttpAPIRequest - -parameter: end_date = 20070930
[INFO] HttpAPIRequest - -parameter: ret_mod_name = Y
[INFO] HttpAPIRequest - -parameter: ret_link_name = Y
[INFO] HttpAPIRequest - -parameter: profile_field_names = CUST_ID
[DEBUG] HttpConnection - -Open connection to xxxx.xxxx.xxx:9000
[DEBUG] header - ->> "POST /api/ActivityDownload.mpl HTTP/1.1[\r][\n]"
[DEBUG] HttpMethodBase - -Adding Host request header
[DEBUG] HttpMethodBase - -Default charset used: ISO-8859-1
[DEBUG] HttpMethodBase - -Default charset used: ISO-8859-1
[DEBUG] header - ->> "User-Agent: Mozilla/5.0 (Windows; U; Windows NT
5.1; en-US; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7[\r][\n]"
[DEBUG] header - ->> "Keep-Alive: 300[\r][\n]"
[DEBUG] header - ->> "Connection: keep-alive[\r][\n]"
[DEBUG] header - ->> "Accept:
text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plai
n;q=0.8,image/png,*/*;q=0.5[\r][\n]"
[DEBUG] header - ->> "Accept-Language: en-us,en;q=0.5[\r][\n]"
[DEBUG] header - ->> "Accept-Encoding: gzip,deflate[\r][\n]"
[DEBUG] header - ->> "Accept-Charset:
ISO-8859-1,utf-8;q=0.7,*;q=0.7[\r][\n]"
[DEBUG] header - ->> "Host: xxxx.xxx.xxx:9000[\r][\n]"
[DEBUG] header - ->> "Content-Length: 238[\r][\n]"
[DEBUG] header - ->> "Content-Type:
application/x-www-form-urlencoded[\r][\n]"
[DEBUG] header - ->> "[\r][\n]"
[DEBUG] EntityEnclosingMethod - -Request body sent
[DEBUG] header - -<<
"test01||email2@foo.com||email2@foo.com||HVWB||H||G||DEF||||||||1||20070
911 111040||[\r][\n]"
[DEBUG] header - -<<
"test01||email3@foo.com||email3@foo.com||HVWB||H||G||DEF||||||||1||20070
911 111602||[\r][\n]"
[DEBUG] header - -<<
"test01||email3@foo.com||email3@foo.com||HVWB||H||G||DEF||||||||0||20070
911 111602||[\r][\n]"
[DEBUG] header - -<<
"test01||email3@foo.com||email3@foo.com||HVWB||H||G||DEF||||||||0||20070
911 111602||[\r][\n]"
[DEBUG] header - -<<
"test01||email3@foo.com||email3@foo.com||HVWB||H||G||DEF||||||||0||20070
911 111603||[\r][\n]"
[DEBUG] header - -<<
"test01||email2@foo.com||email2@foo.com||SEND||H||S||1||||||||1||2007091
1 110746||[\r][\n]"
[DEBUG] header - -<<
"test01||email4@foo.com||email4@foo.com||SEND||H||S||1||||||||1||2007091
1 110748||[\r][\n]"
[DEBUG] header - -<<
"test01||email3@foo.com||email3@foo.com||SEND||H||S||1||||||||1||2007091
1 110749||[\r][\n]"
[DEBUG] header - -<<
"test01||Email1@foo.com||email1@foo.com||SEND||H||S||1||||||||1||2007091
1 110751||[\r][\n]"
[DEBUG] header - -<<
"test01||email5@foo.com||email5@foo.com||SEND||H||S||1||||||||1||2007091
1 110752||[\r][\n]"
[DEBUG] header - -<<
"test01||email2@foo.com||email2@foo.com||HVWB||H||R||280988||http://www.
epsiloninteractive.com||TOPLEVEL||test_linkname_html||1||20070911
111120||[\r][\n]"
[DEBUG] header - -<<
"test01||Email1@foo.com||email1@foo.com||HVWB||H||G||DEF||||||||0||20070
912 152737||[\r][\n]"
[DEBUG] header - -<<
"test01||Email1@foo.com||email1@foo.com||HVWB||H||R||280988||http://www.
epsiloninteractive.com||TOPLEVEL||test_linkname_html||0||20070912
152742||[\r][\n]"
[DEBUG] header - -<<
"test01||Email1@foo.com||email1@foo.com||HVWB||H||G||DEF||||||||1||20070
913 221549||[\r][\n]"
[DEBUG] header - -<<
"test01||Email1@foo.com||email1@foo.com||HVWB||H||R||280988||http://www.
epsiloninteractive.com||TOPLEVEL||test_linkname_html||1||20070913
221553||[\r][\n]"
[DEBUG] header - -<<
"test01||Email1@foo.com||email1@foo.com||HVWB||H||R||280988||http://www.
epsiloninteractive.com||TOPLEVEL||test_linkname_html||0||20070913
221602||[\r][\n]"
[DEBUG] header - -<<
"test01||Email1@foo.com||email1@foo.com||HVWB||H||G||DEF||||||||0||20070
914 102710||[\r][\n]"
[DEBUG] header - -<<
"test01||Email1@foo.com||email1@foo.com||HVWB||H||G||DEF||||||||0||20070
918 165151||[\r][\n]"
[DEBUG] header - -<<
"test01||email5@foo.com||email5@foo.com||HVWB||H||G||DEF||||||||1||20070
918 170651||[\r][\n]"
[DEBUG] header - -<<
"test01||email5@foo.com||email5@foo.com||HVWB||H||G||DEF||||||||0||20070
918 170651||[\r][\n]"
[DEBUG] header - -<<
"test01||email4@foo.com||email4@foo.com||HVWB||H||G||DEF||||||||1||20070
918 172650||[\r][\n]"
[DEBUG] header - -<<
"test01||email4@foo.com||email4@foo.com||HVWB||H||G||DEF||||||||0||20070
918 172650||[\r][\n]"
[DEBUG] header - -<<
"test01||email2@foo.com||email2@foo.com||HVWB||H||G||DEF||||||||0||20070
918 172743||[\r][\n]"
[DEBUG] header - -<<
"test01||email4@foo.com||email4@foo.com||HVWB||T||C||DEF||||||||1||20070
918 174925||[\r][\n]"
[DEBUG] header - -<<
"test01||email4@foo.com||email4@foo.com||HVWB||T||O||DEF||||||||1||20070
918 175044||[\r][\n]"
[DEBUG] header - -<<
"test01||email2@foo.com||email2@foo.com||SEND||H||S||1||||||||0||2007091
8 170117||[\r][\n]"
[DEBUG] header - -<<
"test01||email4@foo.com||email4@foo.com||SEND||H||S||1||||||||0||2007091
8 170122||[\r][\n]"
[DEBUG] header - -<<
"test01||email3@foo.com||email3@foo.com||SEND||H||S||1||||||||0||2007091
8 170124||[\r][\n]"
[DEBUG] header - -<<
"test01||Email1@foo.com||email1@foo.com||SEND||H||S||1||||||||0||2007091
8 170125||[\r][\n]"
[DEBUG] header - -<<
"test01||email5@foo.com||email5@foo.com||SEND||H||S||1||||||||0||2007091
8 170126||[\r][\n]"
[DEBUG] header - -<<
"test01||Email1@foo.com||email1@foo.com||HVWB||H||G||DEF||||||||0||20070
918 190149||[\r][\n]"
[DEBUG] header - -<<
"test01||Email1@foo.com||email1@foo.com||HVWB||H||R||280988||http://www.
epsiloninteractive.com||TOPLEVEL||test_linkname_html||0||20070918
190154||[\r][\n]"
[DEBUG] header - -<<
"test01||Email1@foo.com||email1@foo.com||HVWB||H||R||331121||http://www.
epsiloninteractive.com||TEST_MODULE||test_module_link_name_html||1||2007
0918 190158||[\r][\n]"
[DEBUG] header - -<<
"test01||Email1@foo.com||email1@foo.com||HVWB||H||G||DEF||||||||0||20070
919 113605||[\r][\n]"
[DEBUG] header - -<< "HTTP/1.1 200 OK[\r][\n]"
[DEBUG] header - -<< "Date: Thu, 27 Sep 2007 15:38:28 GMT[\r][\n]"
[DEBUG] header - -<< "Server: Apache[\r][\n]"
[DEBUG] header - -<< "BFI-Status: OK[\r][\n]"
[DEBUG] header - -<< "BFI-Status-String: [\r][\n]"
[DEBUG] header - -<< "BFI-Number-Records: 34[\r][\n]"
[DEBUG] header - -<< "Connection: close[\r][\n]"
[DEBUG] header - -<< "Transfer-Encoding: chunked[\r][\n]"
[DEBUG] header - -<< "Content-Type: text/plain[\r][\n]"
[WARN] HttpMethodBase - -Going to buffer response body of large or
unknown size. Using getResponseBodyAsStream instead is
recommended.test||

[DEBUG] HttpMethodBase - -Buffering response body
[DEBUG] HttpMethodBase - -Should close connection in response to
directive: close
[DEBUG] HttpConnection - -Releasing connection back to connection
manager.
[INFO] HttpAPIRequest - -postRequest received the following response:
HTTP/1.1 200 OK
[DEBUG] HttpAPIRequest - -response header: Date: Thu, 27 Sep 2007
15:38:28 GMT

[DEBUG] HttpAPIRequest - -response header: Server: Apache

[DEBUG] HttpAPIRequest - -response header: BFI-Status: OK

[DEBUG] HttpAPIRequest - -response header: BFI-Status-String: 

[DEBUG] HttpAPIRequest - -response header: BFI-Number-Records: 34

[DEBUG] HttpAPIRequest - -response header: Connection: close

[DEBUG] HttpAPIRequest - -response header: Transfer-Encoding: chunked

[DEBUG] HttpAPIRequest - -response header: Content-Type: text/plain

[DEBUG] HttpAPIRequest - -response body: 

[INFO] DreamAPIRequest - -BFI-Status: OK
[INFO] DreamAPIRequest - -BFI-Status-String: 
[INFO] DreamAPIRequest - -BFI-Number-Records: 34
[INFO] FileUtility - -Writing string buffer to File:
email_response.datAppend Flag: false

Chris Morris

Sorry about the footer...  I have no control over what the exchange
server appends to my messages on my behalf...

-----Original Message-----
From: Roland Weber [mailto:ossfwot@dubioso.net] 
Sent: Thursday, September 27, 2007 9:33 AM
To: HttpClient User Discussion
Subject: Re: HTTP Response contains headers last

Morris, Chris wrote:
> I'm working with an API that returns the HTTP Response Headers after
the
> response body.  This confuses the HTTPClient and PostMethod into
> thinking that there is no body.

We have code to handle footers in ChunkedInputStream. However,
this is the first time that I encounter footers in the wild,
so maybe that code is buggy. Could you please post a wire log,
including headers and content? Feel free to obfuscate any
sensitive information and cut the contents.
http://jakarta.apache.org/httpcomponents/httpclient-3.x/logging.html

> Is there a way for me to get direct
> access to the response input stream so that I can parse it out myself?

You can hack through the code of course. The socket's stream
is not accessible directly, because the ChunkedInputStream has
to parse the "chunked" encoding. Trailers are only allowed with
chunked encoding. If you have to hack through the code anyway,
you should rather fix what is broken than work around what is
already there.

> The information contained in this communication is
> confidential, and is intended only for the sole use of
> the recipient named above. [...]

Always funny to read these footers on mails sent to public
mailing lists...

cheers,
  Roland


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
The information contained in this communication is
confidential, and is intended only for the sole use of
the recipient named above. If the reader of this
message is not the intended recipient, you are hereby
notified that any dissemination, distribution, or copying
of this communication is strictly prohibited. If you have
received this communication in error, please re-send
this communication to the sender and delete the original
message or any copy of it from your computer system.
Thank you.

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org


Mime
View raw message