hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Southern <south...@scripps.edu>
Subject RE: Truncated output with HttpPost but works with cURL
Date Fri, 19 Aug 2011 14:58:20 GMT
Further details from the HttpResponse Entity:

Response encoding: null
Response content length: -1
Chunked?: true
Content-Type: text/xml; charset=UTF-8


-----Original Message-----
From: kim young ill [mailto:khiemu@googlemail.com] 
Sent: Friday, August 19, 2011 10:02 AM
To: HttpClient User Discussion
Subject: Re: Truncated output with HttpPost but works with cURL

could be that response is gzipped without specifying content-encoding  which leads to wrong
content-length ?

On Fri, Aug 19, 2011 at 3:32 PM, Mark Southern <southern@scripps.edu> wrote:

> Dear Peter,
>
> Unfortunately that's not it. The problem occurs whether or not I 
> include the step to dump to the local file (and also with an explicit 
> flush/close added to the BufferedOutputStream).
>
> I have made available the java source, ids text file and cUrl example 
> if case anyone cares for a further look:
>
> http://chemutils.florida.scripps.edu/httpclient/curl.sh
> http://chemutils.florida.scripps.edu/httpclient/ids.txt
> http://chemutils.florida.scripps.edu/httpclient/HttpClientTest.java
>
> Thank you,
> ~Mark.
>
> -----Original Message-----
> From: Pete Keyes [mailto:]
> Sent: Thursday, August 18, 2011 9:29 PM
> To: HttpClient User Discussion
> Subject: RE: Truncated output with HttpPost but works with cURL
>
> This is just a guess, but I don't see where you've closed the 
> BufferedOutputStream passed to the writeTo() method.
>
> ...Pete Keyes
> Starbucks Coffee Co.
>
> ________________________________________
> From: Mark Southern [southern@scripps.edu]
> Sent: Thursday, August 18, 2011 05:48 PM
> To: httpclient-users@hc.apache.org
> Subject: Truncated output with HttpPost but works with cURL
>
> To whomever can help me!
>
> I am submitting to a web service and receiving an xml file in return. 
> I am using POST b/c this is too large for a GET. If I use cURL then this works.
> However with HttpClient (4.1.2) and the same parameters, I am getting 
> a truncated xml file returned.
>
> Does anyone have an idea what might be wrong? I include the test file 
> but not the whole list of parameters I am submitting as this is a further 350Kb.
>
> import java.io.BufferedInputStream;
> import java.io.BufferedOutputStream;
> import java.io.BufferedReader;
> import java.io.File;
> import java.io.FileInputStream;
> import java.io.FileOutputStream;
> import java.io.InputStream;
> import java.io.InputStreamReader;
> import java.util.ArrayList;
> import java.util.List;
>
> import javax.xml.parsers.SAXParser;
> import javax.xml.parsers.SAXParserFactory;
>
> import org.apache.http.HttpResponse;
> import org.apache.http.NameValuePair;
> import org.apache.http.client.HttpClient;
> import org.apache.http.client.entity.UrlEncodedFormEntity;
> import org.apache.http.client.methods.HttpPost;
> import org.apache.http.impl.client.DefaultHttpClient;
> import org.apache.http.message.BasicNameValuePair;
> import org.xml.sax.helpers.DefaultHandler;
>
> public class HttpClientTest {
>
>        public static void main(String[] args) throws Exception {
>                new HttpClientTest().runTest();
>        }
>
>        public void runTest() throws Exception {
>                List<NameValuePair> list = new ArrayList(50000);
>                list.add(new BasicNameValuePair("db", "pcassay"));
>                list.add(new BasicNameValuePair("dbfrom", "pcassay"));
>                // read in ~50000 ids from file. These are existing 
> uids in the pcassay
>                // database.
>                BufferedReader reader = new BufferedReader(new 
> InputStreamReader(getClass().getClassLoader().getResourceAsStream("ids.txt")));
>                String line = null;
>                while (null != (line = reader.readLine())) {
>                        list.add(new BasicNameValuePair("id", line));
>                }
>                list.add(new BasicNameValuePair("term", 
> "summary[activityoutcomemethod]"));
>                list.add(new BasicNameValuePair("version", "2.0"));
>                list.add(new BasicNameValuePair("email", 
> "...@scripps.edu "));
>                list.add(new BasicNameValuePair("tool", 
> "...@scripps.edu "));
>
>                System.out.println("#NameValuePairs: " + list.size());
>
>                HttpPost post = new HttpPost("
> http://eutils.ncbi.nlm.nih.gov/entrez/eutils/elink.fcgi");
>                UrlEncodedFormEntity entity = new 
> UrlEncodedFormEntity(list);
>                post.setEntity(entity);
>                HttpClient client = new DefaultHttpClient();
>                HttpResponse response = client.execute(post);
>
>                // dump to local file so we can refer to it.
>                File file = File.createTempFile("eutils", ".xml");
>                System.out.println("Copying eUtils stream to: " + file);
>                response.getEntity().writeTo(new 
> BufferedOutputStream(new FileOutputStream(file)));
>                InputStream is = new BufferedInputStream(new 
> FileInputStream(file));
>
>                // attempt to parse with sax. There will be an error if 
> only a fragment
>                // of the xml is returned.
>                SAXParserFactory factory = SAXParserFactory.newInstance();
>                SAXParser saxParser = factory.newSAXParser();
>                saxParser.parse(is, new DefaultHandler() {
>                });
>        }
> }
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org


Mime
View raw message