hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erxiang Liu <erxi...@us.ibm.com>
Subject Re: why https site returns 403 when using proxy server?
Date Thu, 04 Jan 2007 20:27:07 GMT

Hi, Roland:

Again, thanks for the quick response! I am amazed you figure out the
product name with the limited information.

>Not yet. But I don't trust the error message. Is bugs.eclipse.org
>running an IBM HTTP Server or is your proxy generating a misleading
>error message?
I think mostly it is a misleading message. I tried another https site(
https://www.adobe.com/products/reader/),
same error, I also tried use apache server. this time, I got 405 response
code instead of 403 when using IBM http server.
But I believe the problem is essentially the same. I downloaded the apache
http server 2.2.3 from
the apache site. I uncommented a few  proxy related configuration in the
httpd.conf file:

LoadModule proxy_module modules/mod_proxy.so
#LoadModule proxy_ajp_module modules/mod_proxy_ajp.so
#LoadModule proxy_balancer_module modules/mod_proxy_balancer.so
LoadModule proxy_connect_module modules/mod_proxy_connect.so
LoadModule proxy_http_module modules/mod_proxy_http.so
#LoadModule proxy_ftp_module modules/mod_proxy_ftp.so

sounds right? I did test it with a http site and it works fine.

>Have you made sure that the proxy requires only NTLMv1 and not NTLMv2?
How to find out it needs NTLMv1 or NTLMv2? actually one can access
the apache proxy server I setup without any user and password.

>Have you tried switching the proxy to basic authentication?
could you point out how?

>Please generate and post a wire log.I'll see what I can make of it. If I
can't help, you may
>have to wait a few days until somebody else can jump in.
here is the log connecting to https://www.adobe.com/products/reader/ via a
apache proxy server, let me know whether this is sufficient:
Hopefully, someone else can also throw some good tips.

2007/01/04 13:54:09:171 CST [DEBUG] DefaultHttpParams - Set parameter
http.useragent = Jakarta Commons-HttpClient/3.0
2007/01/04 13:54:09:171 CST [DEBUG] DefaultHttpParams - Set parameter
http.protocol.version = HTTP/1.1
2007/01/04 13:54:09:187 CST [DEBUG] DefaultHttpParams - Set parameter
http.connection-manager.class = class
org.apache.commons.httpclient.SimpleHttpConnectionManager
2007/01/04 13:54:09:187 CST [DEBUG] DefaultHttpParams - Set parameter
http.protocol.cookie-policy = rfc2109
2007/01/04 13:54:09:187 CST [DEBUG] DefaultHttpParams - Set parameter
http.protocol.element-charset = US-ASCII
2007/01/04 13:54:09:187 CST [DEBUG] DefaultHttpParams - Set parameter
http.protocol.content-charset = ISO-8859-1
2007/01/04 13:54:09:187 CST [DEBUG] DefaultHttpParams - Set parameter
http.method.retry-handler =
org.apache.commons.httpclient.DefaultHttpMethodRetryHandler@45024502
2007/01/04 13:54:09:187 CST [DEBUG] DefaultHttpParams - Set parameter
http.dateparser.patterns = [EEE, dd MMM yyyy HH:mm:ss zzz, EEEE, dd-MMM-yy
HH:mm:ss zzz, EEE MMM d HH:mm:ss yyyy, EEE, dd-MMM-yyyy HH:mm:ss z, EEE,
dd-MMM-yyyy HH-mm-ss z, EEE, dd MMM yy HH:mm:ss z, EEE dd-MMM-yyyy HH:mm:ss
z, EEE dd MMM yyyy HH:mm:ss z, EEE dd-MMM-yyyy HH-mm-ss z, EEE dd-MMM-yy
HH:mm:ss z, EEE dd MMM yy HH:mm:ss z, EEE,dd-MMM-yy HH:mm:ss z,
EEE,dd-MMM-yyyy HH:mm:ss z, EEE, dd-MM-yyyy HH:mm:ss z]
2007/01/04 13:54:09:203 CST [DEBUG] HttpClient - Java version: 1.5.0
2007/01/04 13:54:09:203 CST [DEBUG] HttpClient - Java vendor: IBM
Corporation
2007/01/04 13:54:09:203 CST [DEBUG] HttpClient - Java class path:
C:\IBM\eclipsetoolkit\eclipse\startup.jar
2007/01/04 13:54:09:203 CST [DEBUG] HttpClient - Operating system name:
Windows XP
2007/01/04 13:54:09:203 CST [DEBUG] HttpClient - Operating system
architecture: x86
2007/01/04 13:54:09:203 CST [DEBUG] HttpClient - Operating system version:
5.1 build 2600 Service Pack 2
2007/01/04 13:54:09:203 CST [DEBUG] HttpClient - RCP OSGI Service Provider
1.0: RCP OSGI Service Provider allows plugin-based implementations of
KeyStore, TrustManagerFactory and KeyManagerFactory
2007/01/04 13:54:09:203 CST [DEBUG] HttpClient - IBMJSSE2 1.5: IBM JSSE
provider2 (implements IbmX509 key/trust factories, SSLv3, TLSv1)
2007/01/04 13:54:09:203 CST [DEBUG] HttpClient - IBMJCE 1.2: IBMJCE
Provider implements the following: HMAC-SHA1, MD2, MD5, MARS, SHA,
MD2withRSA, MD5withRSA, SHA1withRSA, RSA, SHA1withDSA, RC2, RC4,
Seal)implements the following:
Signature algorithms               : SHA1withDSA, SHA1withRSA, MD5withRSA,
MD2withRSA,
                                       SHA2withRSA, SHA3withRSA,
SHA5withRSA
Cipher algorithms                  : Blowfish, AES, DES, TripleDES,
PBEWithMD2AndDES,
                                       PBEWithMD2AndTripleDES,
PBEWithMD2AndRC2,
                                       PBEWithMD5AndDES,
PBEWithMD5AndTripleDES,
                                       PBEWithMD5AndRC2, PBEWithSHA1AndDES
                                       PBEWithSHA1AndTripleDES,
PBEWithSHA1AndRC2
                                       PBEWithSHAAnd40BitRC2,
PBEWithSHAAnd128BitRC2
                                       PBEWithSHAAnd40BitRC4,
PBEWithSHAAnd128BitRC4
                                       PBEWithSHAAnd2KeyTripleDES,
PBEWithSHAAnd3KeyTripleDES
                                       Mars, RC2, RC4, ARCFOUR
                                       RSA, Seal
Message authentication code (MAC)  : HmacSHA1, HmacSHA256, HmacSHA384,
HmacSHA512, HmacMD2, HmacMD5
Key agreement algorithm            : DiffieHellman
Key (pair) generator               : Blowfish, DiffieHellman, DSA, AES,
DES, TripleDES, HmacMD5,
                                       HmacSHA1, Mars, RC2, RC4, RSA, Seal,
ARCFOUR
Message digest                     : MD2, MD5, SHA-1, SHA-256, SHA-384,
SHA-512
Algorithm parameter generator      : DiffieHellman, DSA
Algorithm parameter                : Blowfish, DiffieHellman, AES, DES,
TripleDES, DSA, Mars,
                                       PBEwithMD5AndDES, RC2
Key factory                        : DiffieHellman, DSA, RSA
Secret key factory                 : Blowfish, AES, DES, TripleDES, Mars,
RC2, RC4, Seal, ARCFOUR
                                       PKCS5Key, PBKDF1 and
PBKDF2(PKCS5Derived Key).
Certificate                        : X.509
Secure random                      : IBMSecureRandom
Key store                          : JCEKS, PKCS12KS (PKCS12), JKS

2007/01/04 13:54:09:203 CST [DEBUG] HttpClient - IBMJGSSProvider 1.5:
IBMJGSSProvider supports Kerberos V5 Mechanism
2007/01/04 13:54:09:203 CST [DEBUG] HttpClient - IBMCertPath 1.1:
IBMCertPath Provider implements the following:
CertificateFactory                : X.509
CertPathValidator              : PKIX
CertStore                      : Collection, LDAP
CertPathBuilder                : PKIX

2007/01/04 13:54:09:203 CST [DEBUG] HttpClient - IBMSASL 1.5: IBM SASL
provider(implements client mechanisms for: DIGEST-MD5, GSSAPI, EXTERNAL,
PLAIN, CRAM-MD5; server mechanisms for: DIGEST-MD5, GSSAPI, CRAM-MD5)
2007/01/04 13:54:09:203 CST [DEBUG] DefaultHttpParams - Set parameter
http.connection.timeout = 30000
2007/01/04 13:54:09:218 CST [DEBUG] MultiThreadedHttpConnectionManager -
HttpConnectionManager.getConnection:  config =
HostConfiguration[host=https://www.adobe.com,
proxyHost=http://x.xx.xx.xxx], timeout = 0
2007/01/04 13:54:09:218 CST [DEBUG] MultiThreadedHttpConnectionManager -
Allocating new connection,
hostConfig=HostConfiguration[host=https://www.adobe.com,
proxyHost=http://x.xx.xx.xxx]
2007/01/04 13:54:09:218 CST [DEBUG] HttpConnection - Open connection to
x.xx.xx.xxx:80
2007/01/04 13:54:09:234 CST [DEBUG] header - >> "CONNECT www.adobe.com:443
HTTP/1.1"
2007/01/04 13:54:09:234 CST [DEBUG] HttpMethodBase - Adding Host request
header
2007/01/04 13:54:09:234 CST [DEBUG] header - >> "User-Agent: Jakarta
Commons-HttpClient/3.0[\r][\n]"
2007/01/04 13:54:09:234 CST [DEBUG] header - >> "Host:
www.adobe.com[\r][\n]"
2007/01/04 13:54:09:234 CST [DEBUG] header - >> "Proxy-Connection:
Keep-Alive[\r][\n]"
2007/01/04 13:54:09:234 CST [DEBUG] header - >> "[\r][\n]"
2007/01/04 13:54:09:250 CST [DEBUG] header - << "HTTP/1.1 405 Method Not
Allowed[\r][\n]"
2007/01/04 13:54:09:250 CST [DEBUG] header - << "Date: Thu, 04 Jan 2007
19:54:09 GMT[\r][\n]"
2007/01/04 13:54:09:250 CST [DEBUG] header - << "Server: Apache/2.2.3
(Win32)[\r][\n]"
2007/01/04 13:54:09:250 CST [DEBUG] header - << "Allow:
GET,HEAD,POST,OPTIONS,TRACE[\r][\n]"
2007/01/04 13:54:09:250 CST [DEBUG] header - << "Content-Length:
235[\r][\n]"
2007/01/04 13:54:09:250 CST [DEBUG] header - << "Content-Type: text/html;
charset=iso-8859-1[\r][\n]"
2007/01/04 13:54:09:250 CST [DEBUG] ConnectMethod - CONNECT status code 405
response code = 405
2007/01/04 13:54:09:250 CST [DEBUG] HttpMethodDirector - CONNECT failed,
fake the response for the original method

 again, here is the excerpt of the code:
client.getHostConfiguration().setProxy("x.xx.xx.xxx", 80); //"x.xx.xx.xxx"
is the IP address of the proxy server
AuthScope as = new AuthScope("x.xx.xx.xxx", 80);
client.getState().setProxyCredentials(
as, new NTCredentials(proxyUser, proxyPassword, InetAddress.getLocalHost
().getHostName(), "xx.xx.xx"));
//proxyUser is the user name to access the proxy server. and proxyPassword
is the password.
int statusCode = client.executeMethod(method);

thanks,

Michelle.



                                                                           
             Roland Weber                                                  
             <http-async@dubio                                             
             so.net>                                                    To 
                                       HttpClient User Discussion          
             01/03/2007 11:51          <httpclient-user@jakarta.apache.org 
             PM                        >                                   
                                                                        cc 
                                                                           
             Please respond to                                     Subject 
             "HttpClient User          Re: why https site returns 403 when 
                Discussion"            using proxy server?                 
             <httpclient-user@                                             
             jakarta.apache.or                                             
                    g>                                                     
                                                                           
                                                                           
                                                                           




Hello Michelle,

> So when the applications do url.openConnection(), it is calling the
apache
> code, instead of the default JVM url Handler.

I see. Lotus Expeditor?

> Yes, we did try to connect to the https site without the proxy server and
> tried it using the browser.
> both works. Yes. most likely it is the problem in my code as you
suggested.
> Also to note that connecting to a http site via proxy server works fine.

OK. SSL connections over a proxy with NTLM authentication is about
the most complex scenario for connecting that you can get. I dimly
remember some discussions a few years ago, maybe on the old list.
http://mail-archives.apache.org/mod_mbox/jakarta-commons-httpclient-dev/
But I think the problems were resolved.

> I thought  there is a very common user scenario and it should work.

Yes, it should.

> Any idea why the acces is denied?

Not yet. But I don't trust the error message. Is bugs.eclipse.org
running an IBM HTTP Server or is your proxy generating a misleading
error message? Have you made sure that the proxy requires only
NTLMv1 and not NTLMv2? Have you tried switching the proxy to
basic authentication?

> Any suggestions?

Please generate and post a wire log.
http://jakarta.apache.org/commons/httpclient/logging.html

I'll see what I can make of it. If I can't help, you may
have to wait a few days until somebody else can jump in.

cheers,
  Roland


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org


Mime
  • Unnamed multipart/related (inline, None, 0 bytes)
View raw message