hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Kalnichevski <ol...@apache.org>
Subject Re: implementation of a custom HttpRoutePlanner - how to choose the HttpRoute attributes (secure, tunnel type, and layer type)?
Date Mon, 30 Nov 2009 14:43:08 GMT
On Mon, 2009-11-30 at 15:17 +0100, Stefan Wachter wrote:
> Hi Oleg,
> 
> I do not understand your comment: "Which makes perfect sense". You said
> that proxies are usually accessed by http (and not https). You said that
> a route must be marked secure in order to have secure cookies to be
> sent. Now, if I try to use HttpClient with a proxy, connect to this
> proxy using http and mark the route to be secure then I get the
> IllegalStateException.
> 
> Cheers,
> --Stefan
> 

I am sorry but this is going to be my last try.

Proxied HTTPS route takes two hops: (1) initial connection is
established by opening a tunnel with HTTP CONNECT (HTTP and not HTTPS);
(2) once the tunnel is established SSL protocol is layered over the
plain connection. Therefore the route must be both TunnelType.TUNNELLED
and LayerType.LAYERED and can be marked secure. 

Oleg

> 
> Am 30.11.2009 14:38, schrieb Oleg Kalnichevski:
> > On Mon, 2009-11-30 at 10:42 +0100, Stefan Wachter wrote:
> >   
> >> Hi Oleg,
> >>
> >> I implemented a small test case that demonstrates my problem. The
> >> program uses a single proxy that is accessed by http (I use WebScarab).
> >> When the program runs then I get the already mentioned illegal state
> >> exception:
> >>
> >>     
> > Which makes perfect sense.
> >
> >   
> >> planned = HttpRoute[{s}->http://localhost:8008->https://www.gmx.net]
> >> current = HttpRoute[{}->http://localhost:8008->https://www.gmx.net]
> >>         at
> >> org.apache.http.impl.client.DefaultRequestDirector.establishRoute(DefaultRequestDirector.java:672)
> >>         at
> >> org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:385)
> >>         at
> >> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:641)
> >>         at
> >> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:576)
> >>         at
> >> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:554)
> >>         at httpclienttest.Main.main(Main.java:68)
> >>
> >> When I change the boolean tunneledAndLayered from false to true then I
> >> get a SSLPeerUnverifiedException:
> >>
> >>
> >> Exception in thread "main" javax.net.ssl.SSLPeerUnverifiedException:
> >> peer not authenticated
> >>         at
> >> com.sun.net.ssl.internal.ssl.SSLSessionImpl.getPeerCertificates(SSLSessionImpl.java:352)
> >>         at
> >> org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:128)
> >>         at
> >> org.apache.http.conn.ssl.SSLSocketFactory.createSocket(SSLSocketFactory.java:399)
> >>         at
> >> org.apache.http.impl.conn.DefaultClientConnectionOperator.updateSecureConnection(DefaultClientConnectionOperator.java:167)
> >>         at
> >> org.apache.http.impl.conn.AbstractPoolEntry.layerProtocol(AbstractPoolEntry.java:275)
> >>         at
> >> org.apache.http.impl.conn.AbstractPooledConnAdapter.layerProtocol(AbstractPooledConnAdapter.java:122)
> >>         at
> >> org.apache.http.impl.client.DefaultRequestDirector.establishRoute(DefaultRequestDirector.java:668)
> >>         at
> >> org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:385)
> >>         at
> >> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:641)
> >>         at
> >> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:576)
> >>         at
> >> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:554)
> >>         at httpclienttest.Main.main(Main.java:57)
> >>
> >> You said that this exception is caused because the SSL context of the
> >> application is not configured correctly. Unfortunately I do not how the
> >> SSL context is configured. Can you please give me a hint.
> >>
> >>     
> > The official Java SSL tutorial is your best friend
> >
> > Oleg
> >
> >   
> >> Best regards,
> >> --Stefan
> >>
> >>
> >>
> >> package httpclienttest;
> >>
> >> import org.apache.http.HttpException;
> >> import org.apache.http.HttpHost;
> >> import org.apache.http.HttpRequest;
> >> import org.apache.http.client.methods.HttpGet;
> >> import org.apache.http.client.methods.HttpUriRequest;
> >> import org.apache.http.conn.ClientConnectionManager;
> >> import org.apache.http.conn.routing.HttpRoute;
> >> import org.apache.http.conn.routing.HttpRoutePlanner;
> >> import org.apache.http.conn.routing.RouteInfo.LayerType;
> >> import org.apache.http.conn.routing.RouteInfo.TunnelType;
> >> import org.apache.http.conn.scheme.PlainSocketFactory;
> >> import org.apache.http.conn.scheme.Scheme;
> >> import org.apache.http.conn.scheme.SchemeRegistry;
> >> import org.apache.http.conn.ssl.SSLSocketFactory;
> >> import org.apache.http.impl.client.DefaultHttpClient;
> >> import org.apache.http.impl.conn.tsccm.ThreadSafeClientConnManager;
> >> import org.apache.http.params.BasicHttpParams;
> >> import org.apache.http.params.HttpParams;
> >> import org.apache.http.protocol.HttpContext;
> >>
> >> public class Main {
> >>
> >>   public static void main(String[] args) throws Exception {
> >>     HttpParams httpParams = new BasicHttpParams();
> >>
> >>     SchemeRegistry schemeRegistry = new SchemeRegistry();
> >>     schemeRegistry.register(new Scheme("http",
> >> PlainSocketFactory.getSocketFactory(), 80));
> >>     schemeRegistry.register(new Scheme("https",
> >> SSLSocketFactory.getSocketFactory(), 443));
> >>
> >>     ClientConnectionManager connectionManager = new
> >> ThreadSafeClientConnManager(httpParams, schemeRegistry);
> >>
> >>     DefaultHttpClient httpClient = new
> >> DefaultHttpClient(connectionManager, httpParams);
> >>
> >>     HttpRoutePlanner routePlanner = new HttpRoutePlanner() {
> >>
> >>       public HttpRoute determineRoute(HttpHost aTarget, HttpRequest
> >> aRequest, HttpContext aContext) throws HttpException {
> >>
> >>         HttpHost[] proxies = new HttpHost[1];
> >>         proxies[0] = new HttpHost("localhost", 8008, "http");
> >>         boolean isSecure = true;
> >>
> >>         boolean tunneledAndLayered = false;
> >>         TunnelType tunnelType = tunneledAndLayered ?
> >> TunnelType.TUNNELLED : TunnelType.PLAIN;
> >>         LayerType layerType = tunneledAndLayered ? LayerType.LAYERED :
> >> LayerType.PLAIN;
> >>
> >>         return new HttpRoute(aTarget, null, proxies, isSecure,
> >> tunnelType, layerType);
> >>
> >>       }
> >>
> >>     };
> >>
> >>     httpClient.setRoutePlanner(routePlanner);
> >>
> >>     HttpUriRequest request = new HttpGet("https://www.gmx.net");
> >>     httpClient.execute(request);
> >>
> >>   }
> >> }
> >>
> >>
> >> Am 27.11.2009 22:29, schrieb Oleg Kalnichevski:
> >>     
> >>> Stefan Wachter wrote:
> >>>       
> >>>> Hi Oleg,
> >>>>
> >>>> your proposal "you need to mark the route as secure but the initial
> >>>> connection to the proxy should be made via HTTP" does not work. If the
> >>>> route is planned in such a way then an IllegalStateException is raised
> >>>> (as I mentioned in my last post). This is caused by the logic
> >>>> implemented in HttpClient that a secure route can not use an insecure
> >>>> connection.
> >>>>         
> >>> That does not seem to make any sense to me but I did not write HTTP
> >>> route planning code. The default logic seems pretty straightforward.
> >>>
> >>> http://hc.apache.org/httpcomponents-client/httpclient/xref/org/apache/http/impl/conn/DefaultHttpRoutePlanner.html#111
> >>>
> >>>
> >>> Please double-check your code and if you are convinced this is a bug
> >>> in HttpClient please try to reproduce the problem with a test case.
> >>>
> >>> Oleg
> >>>
> >>>       
> >>>> Can you please give me a hint how the SSL context of the application
can
> >>>> be configured correctly?
> >>>>
> >>>> Why I want to implement a custom route planner: I have integrated
> >>>> HttpClient into an (web) application framework to allow easy access
to
> >>>> the HTTP protocol. I do not like the standard jvm proxy mechanism
> >>>> because it is configured virtual machine wide and the configuration
must
> >>>> be done on the command line or by setting system properties. Using a
> >>>> configurable custom route planner just would better fit into the overall
> >>>> structure. I did not expect that implementing a custom route planner
> >>>> would be so difficult. In addition, I like the possibilities for
> >>>> connection pooling that HttpClient offers. I thought that in order to
> >>>> configure the pooling for routes it is best to determine the by myself.
> >>>> Otherwise I must have a close look at the routes that are returned by
> >>>> the "ProxySelectorRoutePlanner".
> >>>>
> >>>> Thank you for your help,
> >>>>
> >>>> --Stefan
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> Am 27.11.2009 17:17, schrieb Oleg Kalnichevski:
> >>>>         
> >>>>> Stefan Wachter wrote:
> >>>>>           
> >>>>>> Hi Oleg,
> >>>>>>
> >>>>>> I still struggle with the implementation of my HttpRoutePlanner.
> >>>>>>
> >>>>>> I try to establish an HTTPS connection to a target host via
a
> >>>>>> proxy. You
> >>>>>> said that "usually the tunnel to the proxy is established using
plain
> >>>>>> HTTP". When I try to return a route where the first hop (the
hop to
> >>>>>> the
> >>>>>> proxy) is using HTTP then the secure flag of the route must
not be
> >>>>>> "true". If you try then the following exception is raised:
> >>>>>>
> >>>>>> java.lang.IllegalStateException: Unable to establish route.
> >>>>>> planned = HttpRoute[{s}->http://localhost:8008->https://www.gmx.net]
> >>>>>> current = HttpRoute[{}->http://localhost:8008->https://www.gmx.net]
> >>>>>>
> >>>>>> I tracked the reason down and found that the isSecure method
of the
> >>>>>> PlainSocketFactory always returns false. This means that if
the
> >>>>>> proxy is
> >>>>>> accessed using http then the route must not be flagged to be
> >>>>>> secure. You
> >>>>>> said that if a route is flagged unsecure then "this will prevent
> >>>>>> HttpClient from sending cookies marked as secure".
> >>>>>>
> >>>>>> To summarize: If I want to contact a target host via a proxy
by https
> >>>>>> and want to have cookies that are marked to be secure to be
sent
> >>>>>> then I
> >>>>>> have to use https to contact the proxy and mark the route as
being
> >>>>>> secure. Right?
> >>>>>>             
> >>>>> Yes, you need to mark the route as secure but the initial connection
> >>>>> to the proxy should be made via HTTP. I have not come across an
HTTP
> >>>>> proxy that supported CONNECT method via HTTPS.
> >>>>>
> >>>>>           
> >>>>>> In your last response you wrote, that for proxied https connections
> >>>>>> TunnelType.TUNELLED and LayerType.LAYERED should be choosen.
When I
> >>>>>> return a route that uses https to access the proxy and the target
> >>>>>> host,
> >>>>>> that has its secure flag set to true and that is tunneled and
> >>>>>> layered, I
> >>>>>> get the following exception:
> >>>>>>
> >>>>>>             
> >>>>> This problem has nothing to do with the route computation or even
> >>>>> HttpClient at all. The SSL context used by your application has
not
> >>>>> been configured correctly.
> >>>>>
> >>>>>           
> >>>>>> javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
> >>>>>>         at
> >>>>>> com.sun.net.ssl.internal.ssl.SSLSessionImpl.getPeerCertificates(SSLSessionImpl.java:352)
> >>>>>>
> >>>>>>
> >>>>>>         at
> >>>>>> org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:128)
> >>>>>>
> >>>>>>
> >>>>>>         at
> >>>>>> org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:339)
> >>>>>>
> >>>>>>
> >>>>>>         at
> >>>>>> org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:123)
> >>>>>>
> >>>>>>
> >>>>>>         at
> >>>>>> org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:147)
> >>>>>>
> >>>>>>
> >>>>>>         at
> >>>>>> org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:101)
> >>>>>>
> >>>>>>
> >>>>>>         at
> >>>>>> org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:381)
> >>>>>>
> >>>>>>
> >>>>>>         at
> >>>>>> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:641)
> >>>>>>
> >>>>>>
> >>>>>>         at
> >>>>>> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:576)
> >>>>>>
> >>>>>>
> >>>>>>         at
> >>>>>> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:554)
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> What is going wrong here? Does it mean that the proxy host needs
to
> >>>>>> have
> >>>>>> a certifacate that is signed by a trusted certifaction authority?
> >>>>>>
> >>>>>>             
> >>>>> I have no idea about expectations of your proxy host. Did CONNECT
> >>>>> method succeed?
> >>>>>
> >>>>>           
> >>>>>> Next try: If I choose a route that uses http to access the proxy
host
> >>>>>> and https to access the target host and use TunnelType.PLAIN
and
> >>>>>> LayerType.PLAIN then the route works. 
> >>>>>>             
> >>>>> I suspect SSL/TLS is not being used in this case.
> >>>>>
> >>>>> The route should be marked as TunnelType.TUNELLED and
> >>>>> LayerType.LAYERED and the SSL context of your application must be
set
> >>>>> up correctly.
> >>>>>
> >>>>> Why do you need a custom route planner in the first place?
> >>>>>
> >>>>> Oleg
> >>>>>
> >>>>>
> >>>>> Yet, this route can not be marked
> >>>>>           
> >>>>>> to be secure. This means that secure cookies are not sent!
> >>>>>>
> >>>>>> Thanks for you patience and help,
> >>>>>>
> >>>>>> --Stefan
> >>>>>>
> >>>>>>             
> >>>>> ---------------------------------------------------------------------
> >>>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> >>>>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
> >>>>>
> >>>>>           
> >>>>
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> >>>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
> >>>>
> >>>>         
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> >>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
> >>>
> >>>       
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> >> For additional commands, e-mail: httpclient-users-help@hc.apache.org
> >>
> >>     
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> > For additional commands, e-mail: httpclient-users-help@hc.apache.org
> >
> >   
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
> 



---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org


Mime
View raw message