hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Henrich Kraemer <henrich.krae...@us.ibm.com>
Subject Re: Preemptive authentication throws IllegalStateException using ISA proxy server
Date Tue, 11 Nov 2008 18:32:54 GMT
Oleg,

Thanks for your answers. I finally got back to this. See my comments below.

This is quite a lengthy response, but I hope this will clarify what I am
after..

Thanks,

Henrich

Oleg Kalnichevski <olegk@apache.org> wrote on 10/22/2008 11:44:16 AM:

> On Tue, 2008-10-21 at 13:32 -0700, Henrich Kraemer wrote:
> > Oleg,
> >
> > The motivation to enable preemptive authentication was to avoid an
> > extra request/response round trip for servers that require each
> > request to be authenticated.
>
> What for? As long as the connection is reused the overhead of an extra
> request/response round trip is negligible (at least it does not outweigh
> additional security risks)

I disagree. Extra roundtrips are worth avoiding for high latency
connections.

IE and Firefox perform preemptive authentication for both BASIC and
DIGEST. I find it quite reasonable that other applications would aim
for the same trade-off.

It is unclear to me how much risk is added by this.
I am not a security expert, so these statements should be
taken with caution.
- BASIC is not secure unless done over SSL. So I think one can assume
  that it is only offered over SSL when security matters.
- DIGEST is also relatively unsecure. It is not clear to me that
  preemptive authentication increases the risk very much. Compared
  to cases where each request causes a unauthorized response it
  might even improve security as the matching unauthorized response
  does not precede immediately the response.
One added risk is that the client may send Authorization headers
to the wrong protection space (defined in RFC 2617) and thus leaks
information.
However this risk can be eliminated if the notion of the
protection space is properly implemented.

>
> > Our app does not know ahead of time what kind of authentication is
> > needed for the encountered targets and proxy (if any) as these are
> > user configurable.
> >
>
> The thing is _only_ BASIC authentication scheme lends itself to
> preemptive authentication. Unless you are sure the target server accepts
> BASIC, preemptive authentication is pretty much pointless.
>
> > With preemptive authentication enabled the extra request/response pair
> > is only needed for the first request. Once the user provided
> > credentials HttpClient keeps the credentials in the HttpClients' state
> > field. HttpClient uses these from then on as long as the same
> > HttpClient instance is used.
> > This works as described as long as:
> > 1. Not both the target and the proxy require authentication and
> > 2. The proxy is not an ISA proxy using NTLM authentication. A proxy
> > with digest or basic authentication does work.
>
> (Some soft of) preemptive DIGEST authentication works with HttpClient
> 4.0 only.
>
> > In my mind an HttpClient should understand cases when preemptive
> > authentication cannot be used and then ignore preemptive
> > authentication mode.
> >
> > Does this scenario make sense?
>
> Can it be you are confusing credentials caching and preemptive
> authentication?

With preemptive authentication I mean:
1. Setting the Authorization header.
2. Enabling an http client application to do this when it makes sense
given the rules of RFC 2617.

For DIGEST authentication this involves:
a. Providing the protection space for which credentials are requested.
b. In addition to the credentials and their protection space, the
app needs to know the nonce, nonce count and opaque values in order
to form the authorization header.
An authentication session is initiated by an unauthorized response
and updated by a subsequent Authentication-Info header received or
another unauthorized response with stale=TRUE.
Each request send increases the nonce count, which must be tracked
as well.

For BASIC authentication the authentication session does not
carry information needed to form a preemptive authorization
header beyond the associated realm.

HttpClient could choose to support both 1 and
2 such that an application would only have to enable
preemptive authentication and the library would do all needed
tracking of authentication sessions.
I was under the impression that HttpClient 3's
preemptive mode does just this -- at least for BASIC
authentication.

The examples given for preemptive authentication
for HttpClient 3 require the preemptive credentials are known in
advance.
I find this too limiting as it does not allow to implement
apps that implement preemptive authentication in the sense
mentioned as web browsers do.

Is this a limitation of HttpClient 3?

>
> > Are there better ways to avoid the extra request/response?
>
> This extra request/response pair usually has a clear purpose:
> transmitting an authentication challenge, which is necessary to provide
> some degree of security.
>
> > I believe the challenge every request was seen using a vanilla apache
> > httpd 2.x. server. Perhaps some simple server side configuration
> > should be suggested?
> > Is using preemptive authentication mode for this purpose outside of
> > what it was designed for?
> >
>
> Please consider moving to HttpClient 4.0 if you need a more flexible
> authentication framework. HttpClient 4.0 can be tweaked to perform
> preemptive authentication using BASIC and partially DIGEST scheme as
> described below:
>

I will look into this more when I get a chance.

Two things I noticed on a quick scan:
AuthScope which is used in CredentialsProvider's
getCredentials(AuthScope authScope) captures the realm.
However for DIGEST authentication the protection
space also involves the domain challenge parameter (See RFC 2617,
section 3.2.1). I am not quite sure what consequences this has if
any.

Also it seems that Authentication-Info headers are not looked
at by the HttpClient 4. Presumably this would have to be done by
the application in some way. Will this cause problems?


> http://svn.apache.
> org/repos/asf/httpcomponents/httpclient/trunk/module-
>
client/src/examples/org/apache/http/examples/client/ClientPreemptiveBasicAuthentication.

> java

> http://svn.apache.
> org/repos/asf/httpcomponents/httpclient/trunk/module-
>
client/src/examples/org/apache/http/examples/client/ClientPreemptiveDigestAuthentication.

> java
>
> Hope this helps
>
> Oleg
>
> > Thanks,
> >
> > Henrich
>
> >
> >
> > Inactive hide details for Oleg Kalnichevski <olegk@apache.org>Oleg
> > Kalnichevski <olegk@apache.org>
> >
> >
> >                                 Oleg Kalnichevski <olegk@apache.org>
> >
> >                                 10/15/2008 10:13 AM
> >                                 Please respond to
> >                                 "HttpClient User Discussion"
> <httpclient-users@hc.apache.org>
> >
> >
> >                To
> >
> > HttpClient User
> > Discussion
> > <httpclient-users@hc.apache.org>
> >
> >                cc
> >
> >
> >
> >           Subject
> >
> > Re: Preemptive
> > authentication
> > throws
> > IllegalStateException using ISA proxy server
> >
> >
> >
> > On Tue, 2008-10-14 at 13:07 -0700, Henrich Kraemer wrote:
> > > Oleg,
> > >
> > > The problem also occurs with HttpClient 3.1. See below for updated
> > log.
> > >
> > > Let me know if you would want me to gather more information.
> > >
> > > Thanks,
> > >
> > > Henrich
> > >
> >
> > Henrich
> >
> > Disable preemptive authentication. It is pointless, as you are not
> > providing default credentials. Besides, you seem to be using NTLM
> > authentication anyways. Only Basic authentication scheme can be used
> > preemptively.
> >
> > Oleg
> >
> >
> > > ...
> > > 11 DEBUG 06:56.34 Set parameter http.useragent = Jakarta
> > > Commons-HttpClient/3.1
> > > 12 DEBUG 06:56.36 Set parameter http.protocol.version = HTTP/1.1
> > > 13 DEBUG 06:56.36 Set parameter http.connection-manager.class =
> > class
> > > org.apache.commons.httpclient.SimpleHttpConnectionManager
> > > 14 DEBUG 06:56.36 Set parameter http.protocol.cookie-policy =
> > default
> > > 15 DEBUG 06:56.36 Set parameter http.protocol.element-charset =
> > US-ASCII
> > > 16 DEBUG 06:56.36 Set parameter http.protocol.content-charset =
> > ISO-8859-1
> > > 17 DEBUG 06:56.36 Set parameter http.method.retry-handler =
> > > org.apache.commons.httpclient.DefaultHttpMethodRetryHandler@44fe44fe
> > > 18 DEBUG 06:56.37 Set parameter http.dateparser.patterns = [EEE, dd
> > MMM
> > > yyyy HH:mm:ss zzz, EEEE, dd-MMM-yy HH:mm:ss zzz, EEE MMM d HH:mm:ss
> > yyyy,
> > > EEE, dd-MMM-yyyy HH:mm:ss z, EEE, dd-MMM-yyyy HH-mm-ss z, EEE, dd
> > MMM yy
> > > HH:mm:ss z, EEE dd-MMM-yyyy HH:mm:ss z, EEE dd MMM yyyy HH:mm:ss z,
> > EEE
> > > dd-MMM-yyyy HH-mm-ss z, EEE dd-MMM-yy HH:mm:ss z, EEE dd MMM yy
> > HH:mm:ss z,
> > > EEE,dd-MMM-yy HH:mm:ss z, EEE,dd-MMM-yyyy HH:mm:ss z, EEE,
> > dd-MM-yyyy
> > > HH:mm:ss z]
> > > 19 DEBUG 06:56.37 Java version: 1.5.0
> > > 20 DEBUG 06:56.37 Java vendor: IBM Corporation
> > > 21 DEBUG 06:56.39 Java class path: C:\AD\Target\e_33GA\eclipse
> > \plugins
> > > \org.eclipse.equinox.launcher_1.0.0.v20070606.jar
> > > 22 DEBUG 06:56.39 Operating system name: Windows XP
> > > 23 DEBUG 06:56.39 Operating system architecture: x86
> > > 24 DEBUG 06:56.39 Operating system version: 5.1 build 2600 Service
> > Pack 2
> > > 25 DEBUG 06:56.40 IBMJSSE2 1.5: IBM JSSE provider2 (implements
> > IbmX509
> > > key/trust factories, SSLv3, TLSv1)
> > > 26 DEBUG 06:56.42 IBMJCE 1.2: IBMJCE Provider implements the
> > following:
> > > HMAC-SHA1, MD2, MD5, MARS, SHA, MD2withRSA, MD5withRSA, SHA1withRSA,
> > RSA,
> > > SHA1withDSA, RC2, RC4, Seal)implements the following:
> > > Signature algorithms               : SHA1withDSA, SHA1withRSA,
> > MD5withRSA,
> > > MD2withRSA,
> > >                                        SHA2withRSA, SHA3withRSA,
> > > SHA5withRSA
> > > Cipher algorithms                  : Blowfish, AES, DES, TripleDES,
> > > PBEWithMD2AndDES,
> > >                                        PBEWithMD2AndTripleDES,
> > > PBEWithMD2AndRC2,
> > >                                        PBEWithMD5AndDES,
> > > PBEWithMD5AndTripleDES,
> > >                                        PBEWithMD5AndRC2,
> > PBEWithSHA1AndDES
> > >                                        PBEWithSHA1AndTripleDES,
> > > PBEWithSHA1AndRC2
> > >                                        PBEWithSHAAnd40BitRC2,
> > > PBEWithSHAAnd128BitRC2
> > >                                        PBEWithSHAAnd40BitRC4,
> > > PBEWithSHAAnd128BitRC4
> > >                                        PBEWithSHAAnd2KeyTripleDES,
> > > PBEWithSHAAnd3KeyTripleDES
> > >                                        Mars, RC2, RC4, ARCFOUR
> > >                                        RSA, Seal
> > > Message authentication code (MAC)  : HmacSHA1, HmacSHA256,
> > HmacSHA384,
> > > HmacSHA512, HmacMD2, HmacMD5
> > > Key agreement algorithm            : DiffieHellman
> > > Key (pair) generator               : Blowfish, DiffieHellman, DSA,
> > AES,
> > > DES, TripleDES, HmacMD5,
> > >                                        HmacSHA1, Mars, RC2, RC4,
> > RSA, Seal,
> > > ARCFOUR
> > > Message digest                     : MD2, MD5, SHA-1, SHA-256,
> > SHA-384,
> > > SHA-512
> > > Algorithm parameter generator      : DiffieHellman, DSA
> > > Algorithm parameter                : Blowfish, DiffieHellman, AES,
> > DES,
> > > TripleDES, DSA, Mars,
> > >                                        PBEwithMD5AndDES, RC2
> > > Key factory                        : DiffieHellman, DSA, RSA
> > > Secret key factory                 : Blowfish, AES, DES, TripleDES,
> > Mars,
> > > RC2, RC4, Seal, ARCFOUR
> > >                                        PKCS5Key, PBKDF1 and PBKDF2
> > > (PKCS5Derived Key).
> > > Certificate                        : X.509
> > > Secure random                      : IBMSecureRandom
> > > Key store                          : JCEKS, PKCS12KS (PKCS12), JKS
> > >
> > > 27 DEBUG 06:56.42 IBMJGSSProvider 1.5: IBMJGSSProvider supports
> > Kerberos V5
> > > Mechanism
> > > 28 DEBUG 06:56.42 IBMCertPath 1.1: IBMCertPath Provider implements
> > the
> > > following:
> > > CertificateFactory                : X.509
> > > CertPathValidator              : PKIX
> > > CertStore                      : Collection, LDAP
> > > CertPathBuilder                : PKIX
> > >
> > > 29 DEBUG 06:56.42 IBMSASL 1.5: IBM SASL provider(implements client
> > > mechanisms for: DIGEST-MD5, GSSAPI, EXTERNAL, PLAIN, CRAM-MD5;
> > server
> > > mechanisms for: DIGEST-MD5, GSSAPI, CRAM-MD5)
> > > 30 DEBUG 06:56.43 Set parameter
> > http.authentication.credential-provider =
> > >
> >
com.ibm.cic.common.transports.httpclient.HttpCredentialsProvider@cbe0cbe
> > > 31 DEBUG 06:56.53 Set parameter http.connection-manager.timeout =
> > 30000
> > > 32 DEBUG 06:56.53 Set parameter http.connection.timeout = 30000
> > > 33 DEBUG 06:56.54 Set parameter http.socket.timeout = 30000
> > > 34 DEBUG 06:56.54 Set parameter http.authentication.preemptive =
> > true
> > > 35 DEBUG 06:56.54 Set parameter http.tcp.nodelay = true
> > > 36 DEBUG 07:03.40 Set parameter http.connection-manager.max-per-host
> > =
> > > {HostConfiguration[]=4}
> > > 37 DEBUG 07:03.40 Set parameter http.connection-manager.max-total =
> > 20
> > > 38 DEBUG 07:03.61 enter download
> > > (download:https://co.be.company.com/repo/repository.xml to 'C:
> > \DOCUME~1
> > > \hkraemer\LOCALS~1\Temp\cicdip_hkraemer\1224013328977\nf
> > > \cicURLLrepository.xml28557xml' expectedSize='UNKNOWN'
> > > 39 DEBUG 07:03.81 Set parameter http.method.retry-handler =
> > > com.ibm.cic.common.transports.httpclient.HttpClientDownloadHandler
> > > $MethodRetryHandler@2e282e28
> > > 40 DEBUG 07:03.87 HttpConnectionManager.getConnection:  config =
> > > HostConfiguration[host=https://co.be.company.com,
> > > proxyHost=http://proxy.ra.company.com:8080], timeout = 30000
> > > 41 DEBUG 07:03.87 Allocating new connection,
> > hostConfig=HostConfiguration
> > > [host=https://co.be.company.com,
> > > proxyHost=http://proxy.ra.company.com:8080]
> > > 42 DEBUG 07:03.89 Preemptively sending default basic credentials
> > > 43 DEBUG 07:03.92 Authenticating with BASIC <any
> > > realm>@co.be.company.com:443
> > > 44 WARNING 07:03.92 Required credentials not available for BASIC
> > <any
> > > realm>@co.be.company.com:443
> > > 45 WARNING 07:03.92 Preemptive authentication requested but no
> > default
> > > credentials available
> > > 46 DEBUG 07:03.92 Open connection to proxy.ra.company.com:8080
> > > 47 DEBUG 07:04.26 Preemptively sending default basic credentials
> > > 48 DEBUG 07:04.26 Authenticating with BASIC <any
> > > realm>@proxy.ra.company.com:8080
> > > 49 WARNING 07:04.26 Required proxy credentials not available for
> > BASIC <any
> > > realm>@proxy.ra.company.com:8080
> > > 50 WARNING 07:04.26 Preemptive authentication requested but no
> > default
> > > proxy credentials available
> > > 51 DEBUG 07:04.29 >> "CONNECT co.be.company.com:443 HTTP/1.1"
> > > 52 DEBUG 07:04.29 Adding Host request header
> > > 53 DEBUG 07:04.29 >> "User-Agent: Jakarta
> > Commons-HttpClient/3.1[\r][\n]"
> > > 54 DEBUG 07:04.29 >> "Host: co.be.company.com[\r][\n]"
> > > 55 DEBUG 07:04.29 >> "Proxy-Connection: Keep-Alive[\r][\n]"
> > > 56 DEBUG 07:04.29 >> "[\r][\n]"
> > > 57 DEBUG 07:04.32 << "HTTP/1.1 407 Proxy Authentication Required
> > ( The ISA
> > > Server requires authorization to fulfill the request. Access to the
> > Web
> > > Proxy filter is denied.  )[\r][\n]"
> > > 58 DEBUG 07:04.32 << "HTTP/1.1 407 Proxy Authentication Required
> > ( The ISA
> > > Server requires authorization to fulfill the request. Access to the
> > Web
> > > Proxy filter is denied.  )[\r][\n]"
> > > 59 DEBUG 07:04.32 << "Via: 1.1 BRAT-WIN2K3SRV[\r][\n]"
> > > 60 DEBUG 07:04.32 << "Proxy-Authenticate: Negotiate[\r][\n]"
> > > 61 DEBUG 07:04.32 << "Proxy-Authenticate: Kerberos[\r][\n]"
> > > 62 DEBUG 07:04.34 << "Proxy-Authenticate: NTLM[\r][\n]"
> > > 63 DEBUG 07:04.34 << "Connection: close[\r][\n]"
> > > 64 DEBUG 07:04.36 << "Proxy-Connection: close[\r][\n]"
> > > 65 DEBUG 07:04.36 << "Pragma: no-cache[\r][\n]"
> > > 66 DEBUG 07:04.36 << "Cache-Control: no-cache[\r][\n]"
> > > 67 DEBUG 07:04.36 << "Content-Type: text/html[\r][\n]"
> > > 68 DEBUG 07:04.36 << "Content-Length: 734   [\r][\n]"
> > > 69 DEBUG 07:04.36 << "[\r][\n]"
> > > 70 DEBUG 07:04.39 CONNECT status code 407
> > > 71 DEBUG 07:04.40 Supported authentication schemes in the order of
> > > preference: [ntlm, digest, basic]
> > > 72 INFO 07:04.40 ntlm authentication scheme selected
> > > 73 DEBUG 07:04.40 Using authentication scheme: ntlm
> > > 74 DEBUG 07:04.40 Authorization challenge processed
> > > 75 DEBUG 07:04.42 Proxy authentication scope: NTLM <any
> > > realm>@proxy.ra.company.com:8080
> > > 76 DEBUG 07:04.42 Proxy credentials required
> > > 77 DEBUG 08:12.57 NTLM <any realm>@proxy.ra.company.com:8080 new
> > > credentials given
> > > 78 DEBUG 08:12.57 Should close connection in response to directive:
> > close
> > > 79 DEBUG 08:12.57 Open connection to proxy.ra.company.com:8080
> > > 80 DEBUG 08:12.61 Preemptively sending default basic credentials
> > > 81 DEBUG 08:12.61 Closing the connection.
> > > 82 DEBUG 08:12.61 Releasing connection back to connection manager.
> > > 83 DEBUG 08:12.61 Freeing connection, hostConfig=HostConfiguration
> > > [host=https://co.be.company.com,
> > > proxyHost=http://proxy.ra.company.com:8080]
> > > 84 DEBUG 08:12.61 Adding connection at: 1224013405477
> > > 85 DEBUG 08:12.61 Notifying no-one, there are no waiting threads
> > > 87 ERROR 08:12.62 Unexpected exception
> > >  <exception>
> > >   java.lang.IllegalStateException: Authentication state already
> > initialized
> > >   <stack>org.apache.commons.httpclient.auth.AuthState.setPreemptive
> > > (AuthState.java:120)</stack>
> > >
> > <stack>org.apache.commons.httpclient.HttpMethodDirector.executeConnect
> > > (HttpMethodDirector.java:487)</stack>
> > >
> >
<stack>org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry
> > > (HttpMethodDirector.java:391)</stack>
> > >
> > <stack>org.apache.commons.httpclient.HttpMethodDirector.executeMethod
> > > (HttpMethodDirector.java:171)</stack>
> > >   <stack>org.apache.commons.httpclient.HttpClient.executeMethod
> > > (HttpClient.java:397)</stack>
> > >   <stack>org.apache.commons.httpclient.HttpClient.executeMethod
> > > (HttpClient.java:323)</stack>
> > >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> > For additional commands, e-mail: httpclient-users-help@hc.apache.org
> >
> >
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message