hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Oleg Kalnichevski (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HTTPCLIENT-1742) No connection reuse if response is compressed
Date Wed, 11 May 2016 09:54:12 GMT

    [ https://issues.apache.org/jira/browse/HTTPCLIENT-1742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15279878#comment-15279878
] 

Oleg Kalnichevski commented on HTTPCLIENT-1742:
-----------------------------------------------

I ran this code
{code:java}
CloseableHttpClient client = HttpClients.createDefault();
for (int i = 0; i < 2; i++) {
    HttpGet get = new HttpGet("http://httpbin.org/gzip");
    try (CloseableHttpResponse response = client.execute(get)) {
        EntityUtils.consume(response.getEntity());
    }
}
{code}

and could clearly see the same connection being used for both requests

{noformat}
[DEBUG] RequestAddCookies - CookieSpec selected: best-match
[DEBUG] RequestAuthCache - Auth cache not set in the context
[DEBUG] PoolingHttpClientConnectionManager - Connection request: [route: {}->http://httpbin.org:80][total
kept alive: 0; route allocated: 0 of 2; total allocated: 0 of 20]
[DEBUG] PoolingHttpClientConnectionManager - Connection leased: [id: 0][route: {}->http://httpbin.org:80][total
kept alive: 0; route allocated: 1 of 2; total allocated: 1 of 20]
[DEBUG] MainClientExec - Opening connection {}->http://httpbin.org:80
[DEBUG] HttpClientConnectionOperator - Connecting to httpbin.org/54.175.219.8:80
[DEBUG] HttpClientConnectionOperator - Connection established 192.168.1.121:39993<->54.175.219.8:80
[DEBUG] MainClientExec - Executing request GET /gzip HTTP/1.1
[DEBUG] MainClientExec - Target auth state: UNCHALLENGED
[DEBUG] MainClientExec - Proxy auth state: UNCHALLENGED
[DEBUG] headers - http-outgoing-0 >> GET /gzip HTTP/1.1
[DEBUG] headers - http-outgoing-0 >> Host: httpbin.org
[DEBUG] headers - http-outgoing-0 >> Connection: Keep-Alive
[DEBUG] headers - http-outgoing-0 >> User-Agent: Apache-HttpClient/4.3.5 (java 1.5)
[DEBUG] headers - http-outgoing-0 >> Accept-Encoding: gzip,deflate
[DEBUG] headers - http-outgoing-0 << HTTP/1.1 200 OK
[DEBUG] headers - http-outgoing-0 << Server: nginx
[DEBUG] headers - http-outgoing-0 << Date: Wed, 11 May 2016 09:50:24 GMT
[DEBUG] headers - http-outgoing-0 << Content-Type: application/json
[DEBUG] headers - http-outgoing-0 << Content-Length: 184
[DEBUG] headers - http-outgoing-0 << Connection: keep-alive
[DEBUG] headers - http-outgoing-0 << Content-Encoding: gzip
[DEBUG] headers - http-outgoing-0 << Access-Control-Allow-Origin: *
[DEBUG] headers - http-outgoing-0 << Access-Control-Allow-Credentials: true
[DEBUG] MainClientExec - Connection can be kept alive indefinitely
[DEBUG] PoolingHttpClientConnectionManager - Connection [id: 0][route: {}->http://httpbin.org:80]
can be kept alive indefinitely
[DEBUG] PoolingHttpClientConnectionManager - Connection released: [id: 0][route: {}->http://httpbin.org:80][total
kept alive: 1; route allocated: 1 of 2; total allocated: 1 of 20]
[DEBUG] RequestAddCookies - CookieSpec selected: best-match
[DEBUG] RequestAuthCache - Auth cache not set in the context
[DEBUG] PoolingHttpClientConnectionManager - Connection request: [route: {}->http://httpbin.org:80][total
kept alive: 1; route allocated: 1 of 2; total allocated: 1 of 20]
[DEBUG] PoolingHttpClientConnectionManager - Connection leased: [id: 0][route: {}->http://httpbin.org:80][total
kept alive: 0; route allocated: 1 of 2; total allocated: 1 of 20]
[DEBUG] MainClientExec - Stale connection check
[DEBUG] MainClientExec - Executing request GET /gzip HTTP/1.1
[DEBUG] MainClientExec - Target auth state: UNCHALLENGED
[DEBUG] MainClientExec - Proxy auth state: UNCHALLENGED
[DEBUG] headers - http-outgoing-0 >> GET /gzip HTTP/1.1
[DEBUG] headers - http-outgoing-0 >> Host: httpbin.org
[DEBUG] headers - http-outgoing-0 >> Connection: Keep-Alive
[DEBUG] headers - http-outgoing-0 >> User-Agent: Apache-HttpClient/4.3.5 (java 1.5)
[DEBUG] headers - http-outgoing-0 >> Accept-Encoding: gzip,deflate
[DEBUG] headers - http-outgoing-0 << HTTP/1.1 200 OK
[DEBUG] headers - http-outgoing-0 << Server: nginx
[DEBUG] headers - http-outgoing-0 << Date: Wed, 11 May 2016 09:50:24 GMT
[DEBUG] headers - http-outgoing-0 << Content-Type: application/json
[DEBUG] headers - http-outgoing-0 << Content-Length: 184
[DEBUG] headers - http-outgoing-0 << Connection: keep-alive
[DEBUG] headers - http-outgoing-0 << Content-Encoding: gzip
[DEBUG] headers - http-outgoing-0 << Access-Control-Allow-Origin: *
[DEBUG] headers - http-outgoing-0 << Access-Control-Allow-Credentials: true
[DEBUG] MainClientExec - Connection can be kept alive indefinitely
[DEBUG] PoolingHttpClientConnectionManager - Connection [id: 0][route: {}->http://httpbin.org:80]
can be kept alive indefinitely
[DEBUG] PoolingHttpClientConnectionManager - Connection released: [id: 0][route: {}->http://httpbin.org:80][total
kept alive: 1; route allocated: 1 of 2; total allocated: 1 of 20]
{noformat}

> No connection reuse if response is compressed
> ---------------------------------------------
>
>                 Key: HTTPCLIENT-1742
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1742
>             Project: HttpComponents HttpClient
>          Issue Type: Bug
>          Components: HttpClient
>    Affects Versions: 4.5.2
>         Environment: Linux, Java 8
>            Reporter: Rainer Jung
>
> Trying current JMeter trunk I ran into a problem, that connections were not being reused.
Debugging into it revealed IMHO a HttpClient/HttpCore problem.
> Below you'll find a wire dump of the response headers:
> 2016/05/10 22:44:15 DEBUG - org.apache.http.wire:  << "HTTP/1.1 200 OK[\r][\n]"
> 2016/05/10 22:44:15 DEBUG - org.apache.http.wire:  << "Date: Tue, 10 May 2016 20:44:15
GMT[\r][\n]"
> 2016/05/10 22:44:15 DEBUG - org.apache.http.wire:  << "Server: Apache[\r][\n]"
> 2016/05/10 22:44:15 DEBUG - org.apache.http.wire:  << "Pragma: no-cache[\r][\n]"
> 2016/05/10 22:44:15 DEBUG - org.apache.http.wire:  << "Cache-Control: no-cache,
no-store, must-revalidate[\r][\n]"
> 2016/05/10 22:44:15 DEBUG - org.apache.http.wire:  << "Expires: 0[\r][\n]"
> 2016/05/10 22:44:15 DEBUG - org.apache.http.wire:  << "Pragma: no-cache[\r][\n]"
> 2016/05/10 22:44:15 DEBUG - org.apache.http.wire:  << "Cache-Control: no-cache,
no-store, must-revalidate[\r][\n]"
> 2016/05/10 22:44:15 DEBUG - org.apache.http.wire:  << "Expires: 0[\r][\n]"
> 2016/05/10 22:44:15 DEBUG - org.apache.http.wire:  << "Pragma: no-cache[\r][\n]"
> 2016/05/10 22:44:15 DEBUG - org.apache.http.wire:  << "Cache-Control: no-cache,
no-store, must-revalidate[\r][\n]"
> 2016/05/10 22:44:15 DEBUG - org.apache.http.wire:  << "Expires: 0[\r][\n]"
> 2016/05/10 22:44:15 DEBUG - org.apache.http.wire:  << "Vary: Accept-Encoding[\r][\n]"
> 2016/05/10 22:44:15 DEBUG - org.apache.http.wire:  << "Content-Encoding: gzip[\r][\n]"
> 2016/05/10 22:44:15 DEBUG - org.apache.http.wire:  << "X-Content-Type-Options:
nosniff[\r][\n]"
> 2016/05/10 22:44:15 DEBUG - org.apache.http.wire:  << "X-Frame-Options: sameorigin[\r][\n]"
> 2016/05/10 22:44:15 DEBUG - org.apache.http.wire:  << "Content-Length: 1194[\r][\n]"
> 2016/05/10 22:44:15 DEBUG - org.apache.http.wire:  << "Keep-Alive: timeout=60,
max=9999[\r][\n]"
> 2016/05/10 22:44:15 DEBUG - org.apache.http.wire:  << "Connection: Keep-Alive[\r][\n]"
> 2016/05/10 22:44:15 DEBUG - org.apache.http.wire:  << "Content-Type: text/html;charset=utf-8[\r][\n]"
> 2016/05/10 22:44:15 DEBUG - org.apache.http.wire:  << "[\r][\n]"
> 2016/05/10 22:44:15 DEBUG - org.apache.http.headers: << HTTP/1.1 200 OK
> 2016/05/10 22:44:15 DEBUG - org.apache.http.headers: << Date: Tue, 10 May 2016
20:44:15 GMT
> 2016/05/10 22:44:15 DEBUG - org.apache.http.headers: << Server: Apache
> 2016/05/10 22:44:15 DEBUG - org.apache.http.headers: << Pragma: no-cache
> 2016/05/10 22:44:15 DEBUG - org.apache.http.headers: << Cache-Control: no-cache,
no-store, must-revalidate
> 2016/05/10 22:44:15 DEBUG - org.apache.http.headers: << Expires: 0
> 2016/05/10 22:44:15 DEBUG - org.apache.http.headers: << Pragma: no-cache
> 2016/05/10 22:44:15 DEBUG - org.apache.http.headers: << Cache-Control: no-cache,
no-store, must-revalidate
> 2016/05/10 22:44:15 DEBUG - org.apache.http.headers: << Expires: 0
> 2016/05/10 22:44:15 DEBUG - org.apache.http.headers: << Pragma: no-cache
> 2016/05/10 22:44:15 DEBUG - org.apache.http.headers: << Cache-Control: no-cache,
no-store, must-revalidate
> 2016/05/10 22:44:15 DEBUG - org.apache.http.headers: << Expires: 0
> 2016/05/10 22:44:15 DEBUG - org.apache.http.headers: << Vary: Accept-Encoding
> 2016/05/10 22:44:15 DEBUG - org.apache.http.headers: << Content-Encoding: gzip
> 2016/05/10 22:44:15 DEBUG - org.apache.http.headers: << X-Content-Type-Options:
nosniff
> 2016/05/10 22:44:15 DEBUG - org.apache.http.headers: << X-Frame-Options: sameorigin
> 2016/05/10 22:44:15 DEBUG - org.apache.http.headers: << Content-Length: 1194
> 2016/05/10 22:44:15 DEBUG - org.apache.http.headers: << Keep-Alive: timeout=60,
max=9999
> 2016/05/10 22:44:15 DEBUG - org.apache.http.headers: << Connection: Keep-Alive
> 2016/05/10 22:44:15 DEBUG - org.apache.http.headers: << Content-Type: text/html;charset=utf-8
> As you can see, the response contains a Content-Length header, is not chunked but is
gzip compressed. The server sets a Connection header keep-alive and sends keep-alive values
that would let the connection be reused. But instead DefaultConnectionReuseStrategy.keepAlive()
returns false, because in the code block
>             if (canResponseHaveBody(response)) {
>                 final Header[] clhs = response.getHeaders(HTTP.CONTENT_LEN);
>                 // Do not reuse if not properly content-length delimited
>                 if (clhs.length == 1) {
> the value of clhs.length is 0. So although the wire dump shows a content-length header,
and the request is not chunked, somewhere the content-length header is being removed.
> As soon as I remove the "Accept-Encoding: gzip, deflate" request header, connections
get reused and the content-length header no longer gets removed.
> I expect the class ResponseContentEncoding to be responsible: it removes the header via
> response.removeHeaders("Content-Length");
> in line 141.
> Regards,
> Rainer



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


Mime
View raw message