commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wynand <wol...@gmail.com>
Subject Re: HttpClient 3.0 - Tunneled HTTPS connections through HTTP proxies
Date Tue, 18 Oct 2005 15:37:29 GMT
Yes, thanks alot.
 This isn't good news for me but at least it narrows my options down to just
one ;-)

 On 10/18/05, Oleg Kalnichevski <olegk@apache.org> wrote:
>
> On Tue, Oct 18, 2005 at 04:39:44PM +0200, Wynand wrote:
> > Oleg,
> > I may have stumbled on the cause of this problem by accident. I read
> that
> > there is such an option as "http_access deny CONNECT" in the squid
> > configuration, and that's exactly what httpclient tries to do when it
> tries
> > to make a ssl connection though a proxy. I don't have access to the
> squid
> > configuration, but that's what I'm guessing the problem is. Your
> comments
> > are appreciated.
>
> This is precisely the cause of the problem. To make matters worse the
> version of Squid you are using appears to have an issue with connection
> management. The first time it returns status 407, Proxy-Authenticate and
> Connection: keep-alive headers, which is perfectly ok. However, when
> HttpClient attempts to authenticate using given credentials, the proxy
> simply drops the connection on unsuspected HttpClient:
>
> - >> "CONNECT www.verisign.com:443 <http://www.verisign.com:443> <
> http://www.verisign.com:443/> HTTP/1.1"
> - >> "User-Agent: Jakarta Commons-HttpClient/3.0-rc4[\r][\n]"
> - >> "Host: www.verisign.com[\r][\n <http://www.verisign.com[/r][/n>]"
> - >> "Proxy-Connection: Keep-Alive[\r][\n]"
> - >> "[\r][\n]"
> - << "HTTP/1.0 407 Proxy Authentication Required[\r][\n]"
> - << "Server: Squid/2.4.STABLE6[\r][\n]"
> - << "Mime-Version: 1.0[\r][\n]"
> - << "Date: Tue, 18 Oct 2005 11:27:51 GMT[\r][\n]"
> - << "Content-Type: text/html[\r][\n]"
> - << "Content-Length: 984[\r][\n]"
> - << "Expires: Tue, 18 Oct 2005 11:27:51 GMT[\r][\n]"
> - << "X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0[\r][\n]"
> - << "Proxy-Authenticate: Basic realm="Squid proxy-caching web
> server"[\r][\n]"
> - << "X-Cache: MISS from neutrino.XXXXXXXXX.co.za[\r][\n]"
> - << "Proxy-Connection: keep-alive[\r][\n]"
> - >> "CONNECT www.verisign.com:443 <http://www.verisign.com:443> <
> http://www.verisign.com:443/> HTTP/1.0"
> - >> "User-Agent: Jakarta Commons-HttpClient/3.0-rc4[\r][\n]"
> - >> "Proxy-Connection: Keep-Alive[\r][\n]"
> - >> "Proxy-Authorization: Basic d29sbWFydzp0eXRlbndv[\r][\n]"
> - >> "Host: www.verisign.com[\r][\n <http://www.verisign.com[/r][/n>]"
> - >> "[\r][\n]"
>
> Your only option is to get someone to reconfigure that proxy server (and
> optionally upgrade it from version 2.4.STABLE6 to something slightly
> more modern)
>
> Hope this helps
>
> Oleg
>
>
> > I'm not sure what a wire log is, but here is all the debug info ;-)
> > 2005/10/18 13:28:18:828 CAT [DEBUG] HttpClient - Java version: 1.4.2_08
> > 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Java vendor: Sun
> > Microsystems Inc.
> > 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Java class path:
> > C:\eclipse\workspace\SimpleWebAgent\bin;C:\Projects\java\lib\jericho-
> > html-1.5-dev1.jar;C:\Projects\java\lib\commons-logging-1.0.4.jar
> > ;C:\Projects\java\lib\commons-codec-1.3.jar;C:\Projects\java\commons-
> > httpclient-3.0-rc4\commons-httpclient-3.0-rc4.jar
> > 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Operating system name:
> > Windows 2000
> > 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Operating system
> > architecture: x86
> > 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Operating system
> version:
> > 5.0
> > 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SUN 1.42: SUN (DSA
> > key/parameter generation; DSA signing; SHA-1, MD5 digests; SecureRandom;
> > X.509 certificates; JKS keystore; PKIX CertPathValidator; PKIX
> > CertPathBuilder; LDAP, Collection CertStores)
> > 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunJSSE 1.42: Sun JSSE
> > provider(implements RSA Signatures, PKCS12, SunX509 key/trust factories,
> > SSLv3, TLSv1)
> > 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunRsaSign 1.42: SUN's
> > provider for RSA signatures
> > 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunJCE 1.42: SunJCE
> > Provider (implements DES, Triple DES, AES, Blowfish, PBE,
> Diffie-Hellman,
> > HMAC-MD5, HMAC-SHA1)
> > 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunJGSS 1.0: Sun
> (Kerberos
> > v5)
> > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> > http.useragent = Jakarta Commons-HttpClient/3.0-rc4
> > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> > http.protocol.version = HTTP/1.1
> > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> > http.connection-manager.class = class
> > org.apache.commons.httpclient.SimpleHttpConnectionManager
> > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> > http.protocol.cookie-policy = rfc2109
> > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> > http.protocol.element-charset = US-ASCII
> > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> > http.protocol.content-charset = ISO-8859-1
> > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> > http.method.retry-handler =
> > org.apache.commons.httpclient.DefaultHttpMethodRetryHandler@197a37c
> > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> > http.dateparser.patterns = [EEE, dd MMM yyyy HH:mm:ss zzz, EEEE,
> dd-MMM-yy
> > HH:mm:ss zzz, EEE MMM d HH:mm:ss yyyy, EEE, dd-MMM-yyyy HH:mm:ss z, EEE,
> > dd-MMM-yyyy HH-mm-ss z, EEE, dd MMM yy HH:mm:ss z, EEE dd-MMM-yyyy
> HH:mm:ss
> > z, EEE dd MMM yyyy HH:mm:ss z, EEE dd-MMM-yyyy HH-mm-ss z, EEE dd-MMM-yy
> > HH:mm:ss z, EEE dd MMM yy HH:mm:ss z, EEE,dd-MMM-yy HH:mm:ss z,
> > EEE,dd-MMM-yyyy HH:mm:ss z, EEE, dd-MM-yyyy HH:mm:ss z]
> > 2005/10/18 13:28:19:109 CAT [DEBUG] HttpConnection - Open connection to
> > proxy.XXXXXXXXX.co.za:3128 <http://proxy.XXXXXXXXX.co.za:3128> <
> http://proxy.XXXXXXXXX.co.za:3128>
> > 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "CONNECT
> > www.verisign.com:443 <http://www.verisign.com:443> <
> http://www.verisign.com:443/> HTTP/1.1"
> > 2005/10/18 13:28:19:156 CAT [DEBUG] HttpMethodBase - Adding Host request
> > header
> > 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "User-Agent: Jakarta
> > Commons-HttpClient/3.0-rc4[\r][\n]"
> > 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "Host:
> > www.verisign.com[\r][\n <http://www.verisign.com[/r][/n>]"
> > 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "Proxy-Connection:
> > Keep-Alive[\r][\n]"
> > 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "[\r][\n]"
> > 2005/10/18 13:28:19:156 CAT [DEBUG] header - << "HTTP/1.0 407 Proxy
> > Authentication Required[\r][\n]"
> > 2005/10/18 13:28:19:156 CAT [DEBUG] header - << "Server:
> > Squid/2.4.STABLE6[\r][\n]"
> > 2005/10/18 13:28:19:156 CAT [DEBUG] header - << "Mime-Version: 1.0
> [\r][\n]"
> > 2005/10/18 13:28:19:156 CAT [DEBUG] header - << "Date: Tue, 18 Oct 2005
> > 11:27:51 GMT[\r][\n]"
> > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Content-Type:
> > text/html[\r][\n]"
> > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Content-Length:
> > 984[\r][\n]"
> > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Expires: Tue, 18 Oct
> 2005
> > 11:27:51 GMT[\r][\n]"
> > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "X-Squid-Error:
> > ERR_CACHE_ACCESS_DENIED 0[\r][\n]"
> > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Proxy-Authenticate:
> Basic
> > realm="Squid proxy-caching web server"[\r][\n]"
> > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "X-Cache: MISS from
> > neutrino.XXXXXXXXX.co.za[\r][\n]"
> > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Proxy-Connection:
> > keep-alive[\r][\n]"
> > 2005/10/18 13:28:19:203 CAT [DEBUG] ConnectMethod - CONNECT status code
> 407
> > 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Supported
> > authentication schemes in the order of preference: [ntlm, digest, basic]
> > 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Challenge
> for
> > ntlm authentication scheme not available
> > 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Challenge
> for
> > digest authentication scheme not available
> > 2005/10/18 13:28:19:218 CAT [INFO] AuthChallengeProcessor - basic
> > authentication scheme selected
> > 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Using
> > authentication scheme: basic
> > 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor -
> Authorization
> > challenge processed
> > 2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodDirector - Proxy
> > authentication scope: BASIC 'Squid proxy-caching web
> > server'@proxy.XXXXXXXXX.co.za:3128<http://server'@proxy.XXXXXXXXX.co.za:3128>
> > 2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodBase - Should NOT close
> > connection in response to directive: keep-alive
> > 2005/10/18 13:28:19:218 CAT [DEBUG] HttpConnection - Connection is
> locked.
> > Call to releaseConnection() ignored.
> > 2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodDirector - Authenticating
> with
> > BASIC 'Squid proxy-caching web server'@proxy.XXXXXXXXX.co.za:3128<http://server'@proxy.XXXXXXXXX.co.za:3128>
> > 2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodParams - Credential
> charset
> > not configured, using HTTP element charset
> > 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "CONNECT
> > www.verisign.com:443 <http://www.verisign.com:443> <
> http://www.verisign.com:443/> HTTP/1.0"
> > 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodBase - Adding Host request
> > header
> > 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "User-Agent: Jakarta
> > Commons-HttpClient/3.0-rc4[\r][\n]"
> > 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "Proxy-Connection:
> > Keep-Alive[\r][\n]"
> > 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "Proxy-Authorization:
> Basic
> > d29sbWFydzp0eXRlbndv[\r][\n]"
> > 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "Host:
> > www.verisign.com[\r][\n <http://www.verisign.com[/r][/n>]"
> > 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "[\r][\n]"
> > 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodDirector - Closing the
> > connection.
> > 2005/10/18 13:28:19:234 CAT [INFO] HttpMethodDirector - I/O exception (
> > org.apache.commons.httpclient.NoHttpResponseException) caught when
> > processing request: The server www.verisign.com<http://www.verisign.com>
> > <http://www.verisign.com/>failed to respond
> > 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodDirector - The server
> > www.verisign.com <http://www.verisign.com> <http://www.verisign.com/>
> failed to respond <
> > org.apache.commons.httpclient.NoHttpResponseException: The server
> > www.verisign.com <http://www.verisign.com> <http://www.verisign.com/>
> failed to respond>
> > org.apache.commons.httpclient.NoHttpResponseException: The server
> > www.verisign.com <http://www.verisign.com> <http://www.verisign.com/>
> failed to respond
> > at org.apache.commons.httpclient.HttpMethodBase.readStatusLine(
> > HttpMethodBase.java:1835)
> > at org.apache.commons.httpclient.HttpMethodBase.readResponse(
> > HttpMethodBase.java:1590)
> > at org.apache.commons.httpclient.HttpMethodBase.execute(
> HttpMethodBase.java
> > :995)
> > at org.apache.commons.httpclient.ConnectMethod.execute(
> ConnectMethod.java
> > :144)
> > at org.apache.commons.httpclient.HttpMethodDirector.executeConnect(
> > HttpMethodDirector.java:487)
> > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(
> > HttpMethodDirector.java:388)
> > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(
> > HttpMethodDirector.java:170)
> > at org.apache.commons.httpclient.HttpClient.executeMethod(
> HttpClient.java
> > :396)
> > at org.apache.commons.httpclient.HttpClient.executeMethod(
> HttpClient.java
> > :324)
> > at com.XXXXXXXXX.webagent.TestCase.simplestTest(TestCase.java:43)
> > at com.XXXXXXXXX.webagent.TestCase.main(TestCase.java:21)
> > On 10/18/05, Oleg Kalnichevski <olegk@apache.org> wrote:
> >
> > > On Tue, Oct 18, 2005 at 02:30:24PM +0200, Wynand wrote:
> > > > Hi All,
> > > > I've just started using the commons httpclient 3.0 rc4. It works
> just as
> > > > expected, except for connecting to a HTTPS site through a HTTP proxy
> > > > (Squid/2.4.STABLE6).
> > > > I have tried the the example as per the SSL guide, but to no avail;
> I
> > > get
> > > > the following error :
> > > > 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodDirector - The server
> > > > www.verisign.com <http://www.verisign.com> <http://www.verisign.com>
> <http://www.verisign.com/>
> > > failed to respond <
> > > > org.apache.commons.httpclient.NoHttpResponseException: The server
> > > > www.verisign.com <http://www.verisign.com> <http://www.verisign.com>
> <http://www.verisign.com/>
> > > failed to respond>
> > > > org.apache.commons.httpclient.NoHttpResponseException: The server
> > > > www.verisign.com <http://www.verisign.com> <http://www.verisign.com>
> <http://www.verisign.com/>
> > > failed to respond
> > > > at org.apache.commons.httpclient.HttpMethodBase.readStatusLine (
> > > > HttpMethodBase.java:1835)
> > > > at org.apache.commons.httpclient.HttpMethodBase.readResponse(
> > > > HttpMethodBase.java:1590)
> > > > at org.apache.commons.httpclient.HttpMethodBase.execute(
> > > HttpMethodBase.java
> > > > :995)
> > > > at org.apache.commons.httpclient.ConnectMethod.execute (
> > > ConnectMethod.java
> > > > :144)
> > > > at org.apache.commons.httpclient.HttpMethodDirector.executeConnect(
> > > > HttpMethodDirector.java:487)
> > > > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry
> (
> > > > HttpMethodDirector.java :388)
> > > > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(
> > > > HttpMethodDirector.java:170)
> > > > at org.apache.commons.httpclient.HttpClient.executeMethod(
> > > HttpClient.java
> > > > :396)
> > > > at org.apache.commons.httpclient.HttpClient.executeMethod (
> > > HttpClient.java
> > > > :324)
> > > > at com.wolman.webagent.TestCase.simplestTest(TestCase.java:43)
> > > > at com.wolman.webagent.TestCase.main(TestCase.java:21)
> > > > Just to clarify here is the example i used :
> > > > HttpClient httpclient = new HttpClient();
> > > > httpclient.getHostConfiguration().setProxy("myproxyhost", 8080);
> > > > httpclient.getState().setProxyCredentials("my-proxy-realm", "
> > > myproxyhost",
> > > > new UsernamePasswordCredentials("my-proxy-username",
> > > "my-proxy-password"));
> > > > GetMethod httpget = new GetMethod("*https://www.verisign.com/*");
> > > > httpclient.executeMethod(httpget);
> > > > System.out.println(httpget.getStatusLine().toString());
> > > > It works fine if I change the *https* to *http* in the url. Can
> someone
> > > > please confirm that this example is indeed working or if I'm missing
> > > > something
> > > > Thanks alot
> > >
> > > Wynand,
> > >
> > > Please send the complete wire log.
> > >
> > > Oleg
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail: commons-user-help@jakarta.apache.org
> > >
> > >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-user-help@jakarta.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message