hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pierre N. (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HTTPCLIENT-1550) "deflate" encoding issue : java.util.zip.ZipException: invalid stored block lengths
Date Sat, 28 Nov 2015 14:57:11 GMT

    [ https://issues.apache.org/jira/browse/HTTPCLIENT-1550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15030541#comment-15030541
] 

Pierre N. commented on HTTPCLIENT-1550:
---------------------------------------

Hi Oleg, I think you forgot to consume the stream, if you don't, obviously it won't trigger
the exception because the decompression doesn't occur.

{code}
    @Test
    public void testDeflateIssue() throws Exception {
        String[] websites = new String[]{
            "http://httpbin.org/deflate", // fail
            "https://twitter.com/", // fail
            "http://vk.com/", // fail
            "http://edition.cnn.com/", // fail
            "https://bing.com/" // pass
        };
        
        try(CloseableHttpClient client = HttpClients.createDefault()){
            for (String website : websites) {
                HttpGet request = new HttpGet(website);
                request.setHeader("Accept-Encoding", "deflate"); // make sure server send
deflate and not gzip
                try(CloseableHttpResponse response = client.execute(request)){
                    EntityUtils.toString(response.getEntity());
                }
            }
        }
    }
{code}

{code}
Running httpclient.HttpClient1550
2015/11/28 15:54:47:367 CET [DEBUG] RequestAddCookies - CookieSpec selected: default
2015/11/28 15:54:47:381 CET [DEBUG] RequestAuthCache - Auth cache not set in the context
2015/11/28 15:54:47:382 CET [DEBUG] PoolingHttpClientConnectionManager - Connection request:
[route: {}->http://httpbin.org:80][total kept alive: 0; route allocated: 0 of 2; total
allocated: 0 of 20]
2015/11/28 15:54:47:393 CET [DEBUG] PoolingHttpClientConnectionManager - Connection leased:
[id: 0][route: {}->http://httpbin.org:80][total kept alive: 0; route allocated: 1 of 2;
total allocated: 1 of 20]
2015/11/28 15:54:47:394 CET [DEBUG] MainClientExec - Opening connection {}->http://httpbin.org:80
2015/11/28 15:54:47:422 CET [DEBUG] DefaultHttpClientConnectionOperator - Connecting to httpbin.org/54.175.222.246:80
2015/11/28 15:54:47:529 CET [DEBUG] DefaultHttpClientConnectionOperator - Connection established
10.174.1.6:38798<->54.175.222.246:80
2015/11/28 15:54:47:530 CET [DEBUG] MainClientExec - Executing request GET /deflate HTTP/1.1
2015/11/28 15:54:47:530 CET [DEBUG] MainClientExec - Target auth state: UNCHALLENGED
2015/11/28 15:54:47:530 CET [DEBUG] MainClientExec - Proxy auth state: UNCHALLENGED
2015/11/28 15:54:47:531 CET [DEBUG] headers - http-outgoing-0 >> GET /deflate HTTP/1.1
2015/11/28 15:54:47:531 CET [DEBUG] headers - http-outgoing-0 >> Accept-Encoding: deflate
2015/11/28 15:54:47:532 CET [DEBUG] headers - http-outgoing-0 >> Host: httpbin.org
2015/11/28 15:54:47:532 CET [DEBUG] headers - http-outgoing-0 >> Connection: Keep-Alive
2015/11/28 15:54:47:532 CET [DEBUG] headers - http-outgoing-0 >> User-Agent: Apache-HttpClient/4.5.1
(Java/1.8.0_66)
2015/11/28 15:54:47:640 CET [DEBUG] headers - http-outgoing-0 << HTTP/1.1 200 OK
2015/11/28 15:54:47:640 CET [DEBUG] headers - http-outgoing-0 << Server: nginx
2015/11/28 15:54:47:640 CET [DEBUG] headers - http-outgoing-0 << Date: Sat, 28 Nov 2015
14:54:47 GMT
2015/11/28 15:54:47:641 CET [DEBUG] headers - http-outgoing-0 << Content-Type: application/json
2015/11/28 15:54:47:641 CET [DEBUG] headers - http-outgoing-0 << Content-Length: 172
2015/11/28 15:54:47:641 CET [DEBUG] headers - http-outgoing-0 << Connection: keep-alive
2015/11/28 15:54:47:641 CET [DEBUG] headers - http-outgoing-0 << Content-Encoding: deflate
2015/11/28 15:54:47:641 CET [DEBUG] headers - http-outgoing-0 << Access-Control-Allow-Origin:
*
2015/11/28 15:54:47:641 CET [DEBUG] headers - http-outgoing-0 << Access-Control-Allow-Credentials:
true
2015/11/28 15:54:47:644 CET [DEBUG] MainClientExec - Connection can be kept alive indefinitely
2015/11/28 15:54:47:651 CET [DEBUG] PoolingHttpClientConnectionManager - Connection [id: 0][route:
{}->http://httpbin.org:80] can be kept alive indefinitely
2015/11/28 15:54:47:651 CET [DEBUG] PoolingHttpClientConnectionManager - Connection released:
[id: 0][route: {}->http://httpbin.org:80][total kept alive: 1; route allocated: 1 of 2;
total allocated: 1 of 20]
2015/11/28 15:54:47:651 CET [DEBUG] PoolingHttpClientConnectionManager - Connection manager
is shutting down
2015/11/28 15:54:47:651 CET [DEBUG] DefaultManagedHttpClientConnection - http-outgoing-0:
Close connection
2015/11/28 15:54:47:652 CET [DEBUG] PoolingHttpClientConnectionManager - Connection manager
shut down
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.73 sec <<< FAILURE!
testDeflateIssue(httpclient.HttpClient1550)  Time elapsed: 0.682 sec  <<< ERROR!
java.util.zip.ZipException: invalid stored block lengths
	at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:164)
	at org.apache.http.client.entity.DeflateInputStream.read(DeflateInputStream.java:153)
	at org.apache.http.client.entity.LazyDecompressingInputStream.read(LazyDecompressingInputStream.java:73)
	at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
	at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
	at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
	at java.io.InputStreamReader.read(InputStreamReader.java:184)
	at java.io.Reader.read(Reader.java:140)
	at org.apache.http.util.EntityUtils.toString(EntityUtils.java:247)
	at org.apache.http.util.EntityUtils.toString(EntityUtils.java:291)
	at httpclient.HttpClient1550.testDeflateIssue(HttpClient1550.java:47)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
	at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
	at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
{code}



> "deflate" encoding issue : java.util.zip.ZipException: invalid stored block lengths
> -----------------------------------------------------------------------------------
>
>                 Key: HTTPCLIENT-1550
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1550
>             Project: HttpComponents HttpClient
>          Issue Type: Bug
>    Affects Versions: 4.3.5
>         Environment: Windows
>            Reporter: Guillaume Serre
>            Priority: Minor
>
> When trying to GET any nzb files from http://nzbindex.nl with a httpclient request, a
ZipException is raised.
> {code:title=JunitTestFailing.java}
> 	@Test
> 	public void testDeflateIssueFailing() throws ClientProtocolException, IOException {
> 		HttpGet httpget = new HttpGet("http://nzbindex.nl/download/72441827/BackTrack5R3.Gnome.iSO.32bit-LiNUX-0171-bt5r2gis32.par2.nzb");
> 		HttpClient client = HttpClientBuilder.create().build();
> 		HttpResponse response = client.execute(httpget);
> 		HttpEntity entity = response.getEntity();
> 		byte[] data = EntityUtils.toByteArray( entity );
> 		Assert.assertEquals( data.length, 423548 );
> 	}
> {code}
> The complete stack trace I get is this:
> {code}
> java.util.zip.ZipException: invalid stored block lengths
> 	at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:164)
> 	at java.io.FilterInputStream.read(FilterInputStream.java:107)
> 	at org.apache.http.client.entity.DeflateInputStream.read(DeflateInputStream.java:144)
> 	at org.apache.http.client.entity.LazyDecompressingInputStream.read(LazyDecompressingInputStream.java:68)
> 	at org.apache.http.util.EntityUtils.toByteArray(EntityUtils.java:136)
> 	at hclient.HTTPClientTest.testDeflateIssueFailing(HTTPClientTest.java:39)
> {code}
> A workaround I found is to set the Accept-Encoding header to only accept gzip, as follows:
> {code:title=JunitTestWorking.java}
> 	
> 	@Test
> 	public void testDeflateIssueWorking() throws ClientProtocolException, IOException {
> 		HttpGet httpget = new HttpGet("http://nzbindex.nl/download/72441827/BackTrack5R3.Gnome.iSO.32bit-LiNUX-0171-bt5r2gis32.par2.nzb");
> 		httpget.setHeader("Accept-Encoding", "gzip");
> 		HttpClient client = HttpClientBuilder.create().build();
> 		HttpResponse response = client.execute(httpget);
> 		HttpEntity entity = response.getEntity();
> 		byte[] data = EntityUtils.toByteArray( entity );
> 		Assert.assertEquals( data.length, 423548 );
> 	}
> {code}
> Thanks



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


Mime
View raw message