commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Kalnichevski <ol...@apache.org>
Subject Re: HttpClient 3.0 - Tunneled HTTPS connections through HTTP proxies
Date Tue, 18 Oct 2005 15:23:46 GMT
On Tue, Oct 18, 2005 at 04:39:44PM +0200, Wynand wrote:
> Oleg,
>  I may have stumbled on the cause of this problem by accident. I read that
> there is such an option as "http_access deny CONNECT" in the squid
> configuration, and that's exactly what httpclient tries to do when it tries
> to make a ssl connection though a proxy. I don't have access to the squid
> configuration, but that's what I'm guessing the problem is. Your comments
> are appreciated.

This is precisely the cause of the problem. To make matters worse the
version of Squid you are using appears to have an issue with connection
management. The first time it returns status 407, Proxy-Authenticate and 
Connection: keep-alive headers, which is perfectly ok. However, when 
HttpClient attempts to authenticate using given credentials, the proxy
simply drops the connection on unsuspected HttpClient:

- >> "CONNECT www.verisign.com:443 <http://www.verisign.com:443/> HTTP/1.1"
- >> "User-Agent: Jakarta Commons-HttpClient/3.0-rc4[\r][\n]"
- >> "Host: www.verisign.com[\r][\n <http://www.verisign.com[/r][/n>]"
- >> "Proxy-Connection: Keep-Alive[\r][\n]"
- >> "[\r][\n]"
- << "HTTP/1.0 407 Proxy Authentication Required[\r][\n]"
- << "Server: Squid/2.4.STABLE6[\r][\n]"
- << "Mime-Version: 1.0[\r][\n]"
- << "Date: Tue, 18 Oct 2005 11:27:51 GMT[\r][\n]"
- << "Content-Type: text/html[\r][\n]"
- << "Content-Length: 984[\r][\n]"
- << "Expires: Tue, 18 Oct 2005 11:27:51 GMT[\r][\n]"
- << "X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0[\r][\n]"
- << "Proxy-Authenticate: Basic realm="Squid proxy-caching web server"[\r][\n]"
- << "X-Cache: MISS from neutrino.XXXXXXXXX.co.za[\r][\n]"
- << "Proxy-Connection: keep-alive[\r][\n]"
- >> "CONNECT www.verisign.com:443 <http://www.verisign.com:443/> HTTP/1.0"
- >> "User-Agent: Jakarta Commons-HttpClient/3.0-rc4[\r][\n]"
- >> "Proxy-Connection: Keep-Alive[\r][\n]"
- >> "Proxy-Authorization: Basic d29sbWFydzp0eXRlbndv[\r][\n]"
- >> "Host: www.verisign.com[\r][\n <http://www.verisign.com[/r][/n>]"
- >> "[\r][\n]"

Your only option is to get someone to reconfigure that proxy server (and
optionally upgrade it from version 2.4.STABLE6 to something slightly
more modern)

Hope this helps

Oleg


>  I'm not sure what a wire log is, but here is all the debug info ;-)
>  2005/10/18 13:28:18:828 CAT [DEBUG] HttpClient - Java version: 1.4.2_08
> 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Java vendor: Sun
> Microsystems Inc.
> 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Java class path:
> C:\eclipse\workspace\SimpleWebAgent\bin;C:\Projects\java\lib\jericho-
> html-1.5-dev1.jar;C:\Projects\java\lib\commons-logging-1.0.4.jar
> ;C:\Projects\java\lib\commons-codec-1.3.jar;C:\Projects\java\commons-
> httpclient-3.0-rc4\commons-httpclient-3.0-rc4.jar
> 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Operating system name:
> Windows 2000
> 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Operating system
> architecture: x86
> 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Operating system version:
> 5.0
> 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SUN 1.42: SUN (DSA
> key/parameter generation; DSA signing; SHA-1, MD5 digests; SecureRandom;
> X.509 certificates; JKS keystore; PKIX CertPathValidator; PKIX
> CertPathBuilder; LDAP, Collection CertStores)
> 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunJSSE 1.42: Sun JSSE
> provider(implements RSA Signatures, PKCS12, SunX509 key/trust factories,
> SSLv3, TLSv1)
> 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunRsaSign 1.42: SUN's
> provider for RSA signatures
> 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunJCE 1.42: SunJCE
> Provider (implements DES, Triple DES, AES, Blowfish, PBE, Diffie-Hellman,
> HMAC-MD5, HMAC-SHA1)
> 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunJGSS 1.0: Sun (Kerberos
> v5)
> 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> http.useragent = Jakarta Commons-HttpClient/3.0-rc4
> 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> http.protocol.version = HTTP/1.1
> 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> http.connection-manager.class = class
> org.apache.commons.httpclient.SimpleHttpConnectionManager
> 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> http.protocol.cookie-policy = rfc2109
> 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> http.protocol.element-charset = US-ASCII
> 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> http.protocol.content-charset = ISO-8859-1
> 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> http.method.retry-handler =
> org.apache.commons.httpclient.DefaultHttpMethodRetryHandler@197a37c
> 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> http.dateparser.patterns = [EEE, dd MMM yyyy HH:mm:ss zzz, EEEE, dd-MMM-yy
> HH:mm:ss zzz, EEE MMM d HH:mm:ss yyyy, EEE, dd-MMM-yyyy HH:mm:ss z, EEE,
> dd-MMM-yyyy HH-mm-ss z, EEE, dd MMM yy HH:mm:ss z, EEE dd-MMM-yyyy HH:mm:ss
> z, EEE dd MMM yyyy HH:mm:ss z, EEE dd-MMM-yyyy HH-mm-ss z, EEE dd-MMM-yy
> HH:mm:ss z, EEE dd MMM yy HH:mm:ss z, EEE,dd-MMM-yy HH:mm:ss z,
> EEE,dd-MMM-yyyy HH:mm:ss z, EEE, dd-MM-yyyy HH:mm:ss z]
> 2005/10/18 13:28:19:109 CAT [DEBUG] HttpConnection - Open connection to
> proxy.XXXXXXXXX.co.za:3128 <http://proxy.XXXXXXXXX.co.za:3128>
> 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "CONNECT
> www.verisign.com:443 <http://www.verisign.com:443/> HTTP/1.1"
> 2005/10/18 13:28:19:156 CAT [DEBUG] HttpMethodBase - Adding Host request
> header
> 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "User-Agent: Jakarta
> Commons-HttpClient/3.0-rc4[\r][\n]"
> 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "Host:
> www.verisign.com[\r][\n <http://www.verisign.com[/r][/n>]"
> 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "Proxy-Connection:
> Keep-Alive[\r][\n]"
> 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "[\r][\n]"
> 2005/10/18 13:28:19:156 CAT [DEBUG] header - << "HTTP/1.0 407 Proxy
> Authentication Required[\r][\n]"
> 2005/10/18 13:28:19:156 CAT [DEBUG] header - << "Server:
> Squid/2.4.STABLE6[\r][\n]"
> 2005/10/18 13:28:19:156 CAT [DEBUG] header - << "Mime-Version: 1.0[\r][\n]"
> 2005/10/18 13:28:19:156 CAT [DEBUG] header - << "Date: Tue, 18 Oct 2005
> 11:27:51 GMT[\r][\n]"
> 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Content-Type:
> text/html[\r][\n]"
> 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Content-Length:
> 984[\r][\n]"
> 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Expires: Tue, 18 Oct 2005
> 11:27:51 GMT[\r][\n]"
> 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "X-Squid-Error:
> ERR_CACHE_ACCESS_DENIED 0[\r][\n]"
> 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Proxy-Authenticate: Basic
> realm="Squid proxy-caching web server"[\r][\n]"
> 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "X-Cache: MISS from
> neutrino.XXXXXXXXX.co.za[\r][\n]"
> 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Proxy-Connection:
> keep-alive[\r][\n]"
> 2005/10/18 13:28:19:203 CAT [DEBUG] ConnectMethod - CONNECT status code 407
> 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Supported
> authentication schemes in the order of preference: [ntlm, digest, basic]
> 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Challenge for
> ntlm authentication scheme not available
> 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Challenge for
> digest authentication scheme not available
> 2005/10/18 13:28:19:218 CAT [INFO] AuthChallengeProcessor - basic
> authentication scheme selected
> 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Using
> authentication scheme: basic
> 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Authorization
> challenge processed
> 2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodDirector - Proxy
> authentication scope: BASIC 'Squid proxy-caching web
> server'@proxy.XXXXXXXXX.co.za:3128
> 2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodBase - Should NOT close
> connection in response to directive: keep-alive
> 2005/10/18 13:28:19:218 CAT [DEBUG] HttpConnection - Connection is locked.
> Call to releaseConnection() ignored.
> 2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodDirector - Authenticating with
> BASIC 'Squid proxy-caching web server'@proxy.XXXXXXXXX.co.za:3128
> 2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodParams - Credential charset
> not configured, using HTTP element charset
> 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "CONNECT
> www.verisign.com:443 <http://www.verisign.com:443/> HTTP/1.0"
> 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodBase - Adding Host request
> header
> 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "User-Agent: Jakarta
> Commons-HttpClient/3.0-rc4[\r][\n]"
> 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "Proxy-Connection:
> Keep-Alive[\r][\n]"
> 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "Proxy-Authorization: Basic
> d29sbWFydzp0eXRlbndv[\r][\n]"
> 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "Host:
> www.verisign.com[\r][\n <http://www.verisign.com[/r][/n>]"
> 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "[\r][\n]"
> 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodDirector - Closing the
> connection.
> 2005/10/18 13:28:19:234 CAT [INFO] HttpMethodDirector - I/O exception (
> org.apache.commons.httpclient.NoHttpResponseException) caught when
> processing request: The server www.verisign.com
> <http://www.verisign.com/>failed to respond
> 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodDirector - The server
> www.verisign.com <http://www.verisign.com/> failed to respond <
> org.apache.commons.httpclient.NoHttpResponseException: The server
> www.verisign.com <http://www.verisign.com/> failed to respond>
> org.apache.commons.httpclient.NoHttpResponseException: The server
> www.verisign.com <http://www.verisign.com/> failed to respond
> at org.apache.commons.httpclient.HttpMethodBase.readStatusLine(
> HttpMethodBase.java:1835)
> at org.apache.commons.httpclient.HttpMethodBase.readResponse(
> HttpMethodBase.java:1590)
> at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java
> :995)
> at org.apache.commons.httpclient.ConnectMethod.execute(ConnectMethod.java
> :144)
> at org.apache.commons.httpclient.HttpMethodDirector.executeConnect(
> HttpMethodDirector.java:487)
> at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(
> HttpMethodDirector.java:388)
> at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(
> HttpMethodDirector.java:170)
> at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java
> :396)
> at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java
> :324)
> at com.XXXXXXXXX.webagent.TestCase.simplestTest(TestCase.java:43)
> at com.XXXXXXXXX.webagent.TestCase.main(TestCase.java:21)
>       On 10/18/05, Oleg Kalnichevski <olegk@apache.org> wrote:
> 
> > On Tue, Oct 18, 2005 at 02:30:24PM +0200, Wynand wrote:
> > > Hi All,
> > > I've just started using the commons httpclient 3.0 rc4. It works just as
> > > expected, except for connecting to a HTTPS site through a HTTP proxy
> > > (Squid/2.4.STABLE6).
> > > I have tried the the example as per the SSL guide, but to no avail; I
> > get
> > > the following error :
> > > 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodDirector - The server
> > > www.verisign.com <http://www.verisign.com> <http://www.verisign.com/>
> > failed to respond <
> > > org.apache.commons.httpclient.NoHttpResponseException: The server
> > > www.verisign.com <http://www.verisign.com> <http://www.verisign.com/>
> > failed to respond>
> > > org.apache.commons.httpclient.NoHttpResponseException: The server
> > > www.verisign.com <http://www.verisign.com> <http://www.verisign.com/>
> > failed to respond
> > > at org.apache.commons.httpclient.HttpMethodBase.readStatusLine (
> > > HttpMethodBase.java:1835)
> > > at org.apache.commons.httpclient.HttpMethodBase.readResponse(
> > > HttpMethodBase.java:1590)
> > > at org.apache.commons.httpclient.HttpMethodBase.execute(
> > HttpMethodBase.java
> > > :995)
> > > at org.apache.commons.httpclient.ConnectMethod.execute (
> > ConnectMethod.java
> > > :144)
> > > at org.apache.commons.httpclient.HttpMethodDirector.executeConnect(
> > > HttpMethodDirector.java:487)
> > > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(
> > > HttpMethodDirector.java :388)
> > > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(
> > > HttpMethodDirector.java:170)
> > > at org.apache.commons.httpclient.HttpClient.executeMethod(
> > HttpClient.java
> > > :396)
> > > at org.apache.commons.httpclient.HttpClient.executeMethod (
> > HttpClient.java
> > > :324)
> > > at com.wolman.webagent.TestCase.simplestTest(TestCase.java:43)
> > > at com.wolman.webagent.TestCase.main(TestCase.java:21)
> > > Just to clarify here is the example i used :
> > > HttpClient httpclient = new HttpClient();
> > > httpclient.getHostConfiguration().setProxy("myproxyhost", 8080);
> > > httpclient.getState().setProxyCredentials("my-proxy-realm", "
> > myproxyhost",
> > > new UsernamePasswordCredentials("my-proxy-username",
> > "my-proxy-password"));
> > > GetMethod httpget = new GetMethod("*https://www.verisign.com/*");
> > > httpclient.executeMethod(httpget);
> > > System.out.println(httpget.getStatusLine().toString());
> > > It works fine if I change the *https* to *http* in the url. Can someone
> > > please confirm that this example is indeed working or if I'm missing
> > > something
> > > Thanks alot
> >
> > Wynand,
> >
> > Please send the complete wire log.
> >
> > Oleg
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: commons-user-help@jakarta.apache.org
> >
> >

---------------------------------------------------------------------
To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-user-help@jakarta.apache.org


Mime
View raw message