hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Peter Lancaster (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HTTPCLIENT-1113) Error downloading text file with gzip content encoding
Date Tue, 02 Aug 2011 10:12:27 GMT
Error downloading text file with gzip content encoding
------------------------------------------------------

                 Key: HTTPCLIENT-1113
                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1113
             Project: HttpComponents HttpClient
          Issue Type: Bug
          Components: HttpClient
    Affects Versions: 4.1.1
         Environment: Ubuntu, Java 1.5
            Reporter: Peter Lancaster
            Priority: Critical


Hello I am getting an exception when I try to download certain files.

I don't have control over the host server, only the client.  Here's my client code:

		HttpParams params = new BasicHttpParams();
		params.setParameter(CoreConnectionPNames.CONNECTION_TIMEOUT, 300000L);
		params.setParameter(ClientPNames.HANDLE_REDIRECTS, true);

		// This client indicates to servers that it will support 'gzip'
		// and 'deflate' compressed responses.
		ContentEncodingHttpClient.setDefaultHttpParams(params);
		ContentEncodingHttpClient client = new ContentEncodingHttpClient();

		if (user != null && password != null) {
			String hostname = url.getHost();
			HttpHost hostHttp = new HttpHost(hostname, 80, "http");
			HttpHost hostHttps = new HttpHost(hostname, 443, "https");
			client.getCredentialsProvider().setCredentials(
			        new AuthScope(hostname, 80), 
			        new UsernamePasswordCredentials(user, password));
	
			client.getCredentialsProvider().setCredentials(
			        new AuthScope(hostname, 443), 
			        new UsernamePasswordCredentials(user, password));
	
			// Create AuthCache instance
			AuthCache authCache = new BasicAuthCache();
			// Generate BASIC scheme object and add it to the local auth cache
			BasicScheme basicAuth = new BasicScheme();
			authCache.put(hostHttp, basicAuth);
			authCache.put(hostHttps, basicAuth);
	
			// Add AuthCache to the execution context
			BasicHttpContext localcontext = new BasicHttpContext();
			localcontext.setAttribute(ClientContext.AUTH_CACHE, authCache);
		}
		HttpGet httpget = new HttpGet(url.toString());
		httpget.setHeader("If-Modified-Since", lastModified);


		HttpResponse response = client.execute(httpget);
		responseCode = response.getStatusLine().getStatusCode();
		HttpEntity entity = response.getEntity();
		if (responseCode == HttpStatus.SC_NOT_MODIFIED) {
			
		} else if (responseCode == HttpStatus.SC_OK && entity != null) {
			outStream = new BufferedOutputStream(new FileOutputStream(outFilename));
			entity.writeTo(outStream);
		}

Here's the log output:

DEBUG [2011-08-02 01:23:01,031] [org.apache.http.impl.conn.SingleClientConnManager:212] Get
connection for route HttpRoute[{}->http://<host>]
DEBUG [2011-08-02 01:23:01,036] [org.apache.http.impl.conn.DefaultClientConnectionOperator:145]
Connecting to <host>/<IP>:80
DEBUG [2011-08-02 01:23:01,057] [org.apache.http.client.protocol.RequestAddCookies:132] CookieSpec
selected: best-match
DEBUG [2011-08-02 01:23:01,057] [org.apache.http.client.protocol.RequestAuthCache:75]   Auth
cache not set in the context
DEBUG [2011-08-02 01:23:01,058] [org.apache.http.impl.client.DefaultRequestDirector:631] 
      Attempt 1 to execute request
DEBUG [2011-08-02 01:23:01,058] [org.apache.http.impl.conn.DefaultClientConnection:264] Sending
request: GET <file> HTTP/1.1
DEBUG [2011-08-02 01:23:01,058] [org.apache.http.impl.conn.Wire:63]     >> "GET <file>
HTTP/1.1[\r][\n]"
DEBUG [2011-08-02 01:23:01,058] [org.apache.http.impl.conn.Wire:63]     >> "If-Modified-Since:
Mon, 01 Aug 2011 18:26:09 CEST[\r][\n]"
DEBUG [2011-08-02 01:23:01,059] [org.apache.http.impl.conn.Wire:63]     >> "Host: <host>[\r][\n]"
DEBUG [2011-08-02 01:23:01,059] [org.apache.http.impl.conn.Wire:63]     >> "Connection:
Keep-Alive[\r][\n]"
DEBUG [2011-08-02 01:23:01,059] [org.apache.http.impl.conn.Wire:63]     >> "User-Agent:
Apache-HttpClient/4.1.1 (java 1.5)[\r][\n]"
DEBUG [2011-08-02 01:23:01,059] [org.apache.http.impl.conn.Wire:63]     >> "Accept-Encoding:
gzip,deflate[\r][\n]"
DEBUG [2011-08-02 01:23:01,060] [org.apache.http.impl.conn.Wire:63]     >> "[\r][\n]"
DEBUG [2011-08-02 01:23:01,060] [org.apache.http.impl.conn.DefaultClientConnection:268] >>
GET <file> HTTP/1.1
DEBUG [2011-08-02 01:23:01,060] [org.apache.http.impl.conn.DefaultClientConnection:271] >>
If-Modified-Since: Mon, 01 Aug 2011 18:26:09 CEST
DEBUG [2011-08-02 01:23:01,060] [org.apache.http.impl.conn.DefaultClientConnection:271] >>
Host: <host>
DEBUG [2011-08-02 01:23:01,061] [org.apache.http.impl.conn.DefaultClientConnection:271] >>
Connection: Keep-Alive
DEBUG [2011-08-02 01:23:01,061] [org.apache.http.impl.conn.DefaultClientConnection:271] >>
User-Agent: Apache-HttpClient/4.1.1 (java 1.5)
DEBUG [2011-08-02 01:23:01,061] [org.apache.http.impl.conn.DefaultClientConnection:271] >>
Accept-Encoding: gzip,deflate
DEBUG [2011-08-02 01:23:01,085] [org.apache.http.impl.conn.Wire:63]     << "HTTP/1.1
200 OK[\r][\n]"
DEBUG [2011-08-02 01:23:01,086] [org.apache.http.impl.conn.Wire:63]     << "Server:
nginx/0.8.54[\r][\n]"
DEBUG [2011-08-02 01:23:01,086] [org.apache.http.impl.conn.Wire:63]     << "Date: Mon,
01 Aug 2011 23:23:01 GMT[\r][\n]"
DEBUG [2011-08-02 01:23:01,086] [org.apache.http.impl.conn.Wire:63]     << "Content-Type:
text/plain[\r][\n]"
DEBUG [2011-08-02 01:23:01,086] [org.apache.http.impl.conn.Wire:63]     << "Last-Modified:
Wed, 20 Jul 2011 14:39:57 GMT[\r][\n]"
DEBUG [2011-08-02 01:23:01,087] [org.apache.http.impl.conn.Wire:63]     << "Transfer-Encoding:
chunked[\r][\n]"
DEBUG [2011-08-02 01:23:01,087] [org.apache.http.impl.conn.Wire:63]     << "Connection:
keep-alive[\r][\n]"
DEBUG [2011-08-02 01:23:01,087] [org.apache.http.impl.conn.Wire:63]     << "Vary: Accept-Encoding[\r][\n]"
DEBUG [2011-08-02 01:23:01,087] [org.apache.http.impl.conn.Wire:63]     << "Expires:
Wed, 31 Aug 2011 23:23:01 GMT[\r][\n]"
DEBUG [2011-08-02 01:23:01,088] [org.apache.http.impl.conn.Wire:63]     << "Cache-Control:
max-age=2592000[\r][\n]"
DEBUG [2011-08-02 01:23:01,088] [org.apache.http.impl.conn.Wire:63]     << "Content-Encoding:
gzip[\r][\n]"
DEBUG [2011-08-02 01:23:01,088] [org.apache.http.impl.conn.Wire:63]     << "[\r][\n]"
DEBUG [2011-08-02 01:23:01,088] [org.apache.http.impl.conn.DefaultClientConnection:249] Receiving
response: HTTP/1.1 200 OK
DEBUG [2011-08-02 01:23:01,089] [org.apache.http.impl.conn.DefaultClientConnection:252] <<
HTTP/1.1 200 OK
DEBUG [2011-08-02 01:23:01,089] [org.apache.http.impl.conn.DefaultClientConnection:255] <<
Server: nginx/0.8.54
DEBUG [2011-08-02 01:23:01,089] [org.apache.http.impl.conn.DefaultClientConnection:255] <<
Date: Mon, 01 Aug 2011 23:23:01 GMT
DEBUG [2011-08-02 01:23:01,089] [org.apache.http.impl.conn.DefaultClientConnection:255] <<
Content-Type: text/plain
DEBUG [2011-08-02 01:23:01,089] [org.apache.http.impl.conn.DefaultClientConnection:255] <<
Last-Modified: Wed, 20 Jul 2011 14:39:57 GMT
DEBUG [2011-08-02 01:23:01,090] [org.apache.http.impl.conn.DefaultClientConnection:255] <<
Transfer-Encoding: chunked
DEBUG [2011-08-02 01:23:01,090] [org.apache.http.impl.conn.DefaultClientConnection:255] <<
Connection: keep-alive
DEBUG [2011-08-02 01:23:01,090] [org.apache.http.impl.conn.DefaultClientConnection:255] <<
Vary: Accept-Encoding
DEBUG [2011-08-02 01:23:01,090] [org.apache.http.impl.conn.DefaultClientConnection:255] <<
Expires: Wed, 31 Aug 2011 23:23:01 GMT
DEBUG [2011-08-02 01:23:01,090] [org.apache.http.impl.conn.DefaultClientConnection:255] <<
Cache-Control: max-age=2592000
DEBUG [2011-08-02 01:23:01,091] [org.apache.http.impl.conn.DefaultClientConnection:255] <<
Content-Encoding: gzip
DEBUG [2011-08-02 01:23:01,091] [org.apache.http.impl.client.DefaultRequestDirector:477] 
      Connection can be kept alive indefinitely
DEBUG [2011-08-02 01:23:01,131] [org.apache.http.impl.conn.Wire:63]     << "600a[\r][\n]"
DEBUG [2011-08-02 01:23:01,132] [org.apache.http.impl.conn.Wire:77]     << "[0x1f]"
DEBUG [2011-08-02 01:23:03,838] [org.apache.http.impl.conn.Wire:63]     << "[\r][\n]"

.... (Content)

DEBUG [2011-08-02 01:23:03,839] [org.apache.http.impl.conn.SingleClientConnManager:267] Releasing
connection org.apache.http.impl.conn.SingleClientConnManager$ConnAdapter@2aa3873
DEBUG [2011-08-02 01:23:03,839] [org.apache.http.impl.conn.SingleClientConnManager:285] Released
connection open but not reusable.
DEBUG [2011-08-02 01:23:03,839] [org.apache.http.impl.conn.DefaultClientConnection:152] Connection
shut down
ERROR [2011-08-02 01:23:03,840] [app]        Exception downloading file
java.io.EOFException
        at java.util.zip.GZIPInputStream.readUByte(GZIPInputStream.java:224)
        at java.util.zip.GZIPInputStream.readUShort(GZIPInputStream.java:214)
        at java.util.zip.GZIPInputStream.readHeader(GZIPInputStream.java:153)
        at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:75)
        at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:85)
        at org.apache.http.client.entity.GzipDecompressingEntity.getContent(GzipDecompressingEntity.java:63)
        at org.apache.http.util.EntityUtils.consume(EntityUtils.java:65)
        at org.apache.http.conn.BasicManagedEntity.ensureConsumed(BasicManagedEntity.java:98)
        at org.apache.http.conn.BasicManagedEntity.writeTo(BasicManagedEntity.java:115)
        at util.FileDownload.download(FileDownload.java:188) <--- my app

Does this happen because the server doesn't specify the content length?

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


Mime
View raw message