hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Kalnichevski <ol...@apache.org>
Subject Re: CircularRedirectException while calling URL that works in browsers
Date Fri, 18 Jul 2014 14:25:30 GMT
On Thu, 2014-07-17 at 18:12 +0000, darkimmortal@hushmail.com wrote:
> 
> On Thu, 17 Jul 2014 08:22:10 +0000 "Oleg Kalnichevski" 
> <olegk@apache.org> wrote:
> >On Wed, 2014-07-16 at 23:26 +0000, darkimmortal@hushmail.com 
> >wrote:
> >> Hello,
> >> 
> >> I'm currently trying to call an URL that works fine in IE, 
> >Firefox 
> >> and Chrome but gives me a CircularRedirectException with 
> >HttpClient.
> >> 
> >> First of all, here is a small test case:
> >> 
> >>     public static void main(String[] args) throws 
> >> ClientProtocolException, IOException {
> >> 
> >>             HttpClientBuilder clientBuilder = 
> >HttpClients.custom();
> >>             clientBuilder.setRedirectStrategy(new 
> >> LaxRedirectStrategy());
> >>         
> >>             CloseableHttpClient httpclient = 
> >clientBuilder.build();
> >> 
> >>             String url = 
> >> "http://www.mediafire.com/?yx5bjvfxvkk89hj/Sheena.QOTJ-
> >> 84).part4.rar";
> >> 
> >>             HttpRequestBase request = new HttpGet(url);
> >> 
> >>             httpclient.execute(request);
> >>     }
> >> 
> >> 
> >> After checking with Wireshark, this is apparently what happens:
> >> 
> >> 1. "/?yx5bjvfxvkk89hj/Sheena.QOTJ-84).part4.rar" gets called.
> >> 2. We get a 301 response with the location 
> >> "/download/yx5bjvfxvkk89hj/Sheena.QOTJ-84%29.part4.rar"
> >> 3. "/download/yx5bjvfxvkk89hj/Sheena.QOTJ-84).part4.rar" gets 
> >> called.
> >> 4. We get the same 301 response again.
> >> 
> >> Notice that in step 3 there is a ) in the URL while the response 
> >
> >> specifies %29 instead. Browsers do not make this conversion and 
> >get 
> >> the correct web page instead.
> >> 
> >> Could you kindly tell me if I'm doing anything wrong?
> >> 
> >> Regards,
> >> DSA
> >> 
> >> 
> >
> >Please post a complete wire / context log of the session
> >
> >http://hc.apache.org/httpcomponents-client-4.3.x/logging.html
> >
> >Oleg
> >
> >
> >
> >-------------------------------------------------------------------
> >--
> >To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> >For additional commands, e-mail: httpclient-users-
> >help@hc.apache.org
> 
> Here you go:
> 
> [DEBUG] 17 Jul 2014 20:09:50,086 org.apache.http.wire - http-
> outgoing-0 >> "GET /?yx5bjvfxvkk89hj/Sheena.QOTJ-84).part4.rar 
> HTTP/1.1[\r][\n]"
> [DEBUG] 17 Jul 2014 20:09:50,087 org.apache.http.wire - http-
> outgoing-0 >> "Host: www.mediafire.com[\r][\n]"
> [DEBUG] 17 Jul 2014 20:09:50,087 org.apache.http.wire - http-
> outgoing-0 >> "Connection: Keep-Alive[\r][\n]"
> [DEBUG] 17 Jul 2014 20:09:50,087 org.apache.http.wire - http-
> outgoing-0 >> "User-Agent: Apache-HttpClient/4.3.4 (java 
> 1.5)[\r][\n]"
> [DEBUG] 17 Jul 2014 20:09:50,087 org.apache.http.wire - http-
> outgoing-0 >> "Accept-Encoding: gzip,deflate[\r][\n]"
> [DEBUG] 17 Jul 2014 20:09:50,087 org.apache.http.wire - http-
> outgoing-0 >> "[\r][\n]"
> [DEBUG] 17 Jul 2014 20:09:50,317 org.apache.http.wire - http-
> outgoing-0 << "HTTP/1.1 301[\r][\n]"
> [DEBUG] 17 Jul 2014 20:09:50,317 org.apache.http.wire - http-
> outgoing-0 << "Date: Thu, 17 Jul 2014 18:09:48 GMT[\r][\n]"
> [DEBUG] 17 Jul 2014 20:09:50,317 org.apache.http.wire - http-
> outgoing-0 << "Content-Type: text/html; charset=utf-8[\r][\n]"
> [DEBUG] 17 Jul 2014 20:09:50,317 org.apache.http.wire - http-
> outgoing-0 << "Transfer-Encoding: chunked[\r][\n]"
> [DEBUG] 17 Jul 2014 20:09:50,317 org.apache.http.wire - http-
> outgoing-0 << "Connection: close[\r][\n]"
> [DEBUG] 17 Jul 2014 20:09:50,317 org.apache.http.wire - http-
> outgoing-0 << "Cache-control: no-cache[\r][\n]"
> [DEBUG] 17 Jul 2014 20:09:50,317 org.apache.http.wire - http-
> outgoing-0 << "Expires: 0[\r][\n]"
> [DEBUG] 17 Jul 2014 20:09:50,317 org.apache.http.wire - http-
> outgoing-0 << "Location: /download/yx5bjvfxvkk89hj/Sheena.QOTJ-
> 84%29.part4.rar[\r][\n]"

As you can see the redirect location included in the response is
relative, which is illegal. HttpClient has to normalize the redirect
location URI by rewriting it as absolute. It also removes unnecessary
escape sequences in the process, so '%29' becomes ')'.

You can work the problem around by doing something like that
---
CloseableHttpClient client = HttpClients
        .custom()
        .setRedirectStrategy(new DefaultRedirectStrategy() {
            @Override
            protected URI createLocationURI(final String location)
throws ProtocolException {
                return
super.createLocationURI("http://www.mediafire.com" + location);
            }
        })
        .build();
CloseableHttpResponse response = client.execute(
new HttpGet(

"http://www.mediafire.com/download/yx5bjvfxvkk89hj/Sheena.QOTJ-84).part4.rar"));
try {
    EntityUtils.consume(response.getEntity());
} finally {
    response.close();
}
---

I would strongly suggest fixing at the server side, though.

Oleg


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org


Mime
View raw message