hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mark A. Claassen" <MClaas...@ocie.net>
Subject RE: Proxy Failover
Date Mon, 27 Apr 2015 13:06:04 GMT
Sorry, I wasn't clear.  In dealing with the proxies, I was curious about 2 things.  The first
was how to deal with multiple proxies and failover.  The second was how to deal with authentication.
 

Unless I am mistaken, it works like this:
-- If the proxy is not available, I will get an IOException and could safely resend since
the request since the entities would not have been accessed yet.

-- For an authentication challenge (407), depending on the RequestEntity's answer on isRepeatable(),
this could track back to the code that initiated the request.  The only way to ensure that
the client code wouldn't need to know about this, would be to make sure whatever credentials
are necessary are available before it tries to connect (possible by probing the proxy first).



Mark Claassen
Senior Software Engineer

Donnell Systems, Inc.
130 South Main Street
Leighton Plaza Suite 375
South Bend, IN  46601
E-mail: mailto:mclaassen@ocie.net
Voice: (574)232-3784
Fax: (574)232-4014
  

-----Original Message-----
From: Oleg Kalnichevski [mailto:olegk@apache.org] 
Sent: Saturday, April 25, 2015 7:04 AM
To: HttpClient User Discussion
Subject: Re: Proxy Failover

On Fri, 2015-04-24 at 20:56 +0000, Mark A. Claassen wrote:
> Well, I will keep at it.  I was hoping there would be clear methodology to doing this.
> Can you explain how the 407, authentication required, would work.  In my current example,
I think this would be caught in my determineProxy method and not require a request to be resubmitted.
 Or is this what the RetryHandler was made for?  Maybe this is already handled by the default
implementation?
> 
> Thanks,
> 

Mark, why would you want to black-list a proxy based on a 407 response?
This does not sound right to me.

Anyway, it depends on your specific case to decide what is cheaper: to re-submit the request
on I/O error or to preemptively probe proxies.

Oleg

> Mark Claassen
> Senior Software Engineer
> 
> Donnell Systems, Inc.
> 130 South Main Street
> Leighton Plaza Suite 375
> South Bend, IN  46601
> E-mail: mailto:mclaassen@ocie.net
> Voice: (574)232-3784
> Fax: (574)232-4014
>   
> -----Original Message-----
> From: Oleg Kalnichevski [mailto:olegk@apache.org]
> Sent: Friday, April 24, 2015 10:32 AM
> To: HttpClient User Discussion
> Subject: Re: Proxy Failover
> 
> On Fri, 2015-04-24 at 14:18 +0000, Mark A. Claassen wrote:
> > How do I determine the failure type?  If the connection can't be established, then
it can be considered a proxy failure, but there are other possible errors that would not involve
the proxy.  If I just get a IOException back, how can I be sure of which type of failure it
was?  Also, is this where I would receive the proxy authentication challenge?  If so, wouldn't
it be better to establish the credentials on a separate connection so that the thread making
the request can be oblivious to these details?
> > 
> 
> If you get an I/O error while executing a request via a proxy it is most likely due to
transport to proxy. All other type of errors would be communicated as HTTP messages. I _guess_
it should be fairly safe to black-list proxy on any sort of IOException (except for SSL exceptions).
> 
> Oleg
> 
> > Thanks,
> > 
> > Mark Claassen
> > Senior Software Engineer
> > 
> > Donnell Systems, Inc.
> > 130 South Main Street
> > Leighton Plaza Suite 375
> > South Bend, IN  46601
> > E-mail: mailto:mclaassen@ocie.net
> > Voice: (574)232-3784
> > Fax: (574)232-4014
> >  
> > -----Original Message-----
> > From: Oleg Kalnichevski [mailto:olegk@apache.org]
> > Sent: Friday, April 24, 2015 9:50 AM
> > To: HttpClient User Discussion
> > Subject: Re: Proxy Failover
> > 
> > On Fri, 2015-04-24 at 13:08 +0000, Mark A. Claassen wrote:
> > > Thank you.  I did see this, but saw that is was never finished.
> > > 
> > > I looked into the RetryHandler first, but wasn't sure if that was 
> > > where to put it.  It seemed the RetryHandler stuff was not 
> > > intended to work on non-idempotent operations.  That is why I 
> > > tried to do this in the RoutePlanner, since it would clear that no 
> > > request entities had yet been sent.  (Eventually, I will want to 
> > > send POST requests through the proxy.)
> > > 
> > 
> > It will not be very elegant but one probably could have RetryHandler and RoutePlanner
share state through HttpContext and make RetryHandler 'black-list' proxies upon failure instead
of RetryHandler probing good proxies.
> > 
> > Oleg
> > 
> > 
> > > Mark Claassen
> > > Senior Software Engineer
> > > 
> > > Donnell Systems, Inc.
> > > 130 South Main Street
> > > Leighton Plaza Suite 375
> > > South Bend, IN  46601
> > > E-mail: mailto:mclaassen@ocie.net
> > > Voice: (574)232-3784
> > > Fax: (574)232-4014
> > >   
> > > 
> > > -----Original Message-----
> > > From: Oleg Kalnichevski [mailto:olegk@apache.org]
> > > Sent: Friday, April 24, 2015 6:39 AM
> > > To: HttpClient User Discussion
> > > Subject: Re: Proxy Failover
> > > 
> > > On Thu, 2015-04-23 at 16:45 +0000, Mark A. Claassen wrote:
> > > > I found some information on proxy failover and HttpClient, but not much.
 I was wondering, though, if my approach is OK.  I realize there are inefficiencies, but before
I go there, I want to know if I am even on the right track.
> > > > 
> > > > I extended DefautRoutePlanner and implemented the determineProxy method.
> > > > In that method I get the list of potential proxies and then test them.
 
> > > > I return the first one in the list that works. (Also, when I 
> > > > find one that works, I remember it so I don't need to test it 
> > > > again.)
> > > > 
> > > > In my constructor for my route planner, I create a separate HttpClient
instance that just uses the DefaultRoutePlanner, which returns NULL for determineProxy().
> > > > 
> > > > Has anyone done anything like this?
> > > > 
> > > > Thanks,
> > > > Mark
> > > > 
> > > > 
> > > > 
> > > > 	private boolean testProxy(URI targetURI, Proxy proxy) {
> > > > 			boolean rval;
> > > > 			if (proxy.type() != Proxy.Type.DIRECT) {
> > > > 				InetSocketAddress address = (InetSocketAddress) proxy.address();
> > > > 				HttpHost host = new HttpHost(address.getHostName(), 
> > > > address.getPort(), ApacheNetworkImpl.SCHEME_HTTP);
> > > > 
> > > > 				try {
> > > > 					URI uri = new URI(host.toURI());
> > > > 					RequestBuilder rb = RequestBuilder.get(uri);
> > > > 					HttpUriRequest request = rb.build();
> > > > 					try {
> > > > 						HttpResponse resp = testClient.execute(request);
> > > > 						rval = true;
> > > > 					}
> > > > 					catch (IOException ex) {
> > > > 						rval = false;
> > > > 						connectFailed(targetURI, address, ex);
> > > > 					}
> > > > 					finally {
> > > > 						request.abort();
> > > > 					}
> > > > 				}
> > > > 				catch (URISyntaxException ex) {
> > > > 					rval = false;
> > > > 				}
> > > > 			}
> > > > 			else
> > > > 				rval = true;
> > > > 			return rval;
> > > > 		}
> > > > 
> > > 
> > > Mark,
> > > 
> > > There is a JIRA for similar feature request
> > > 
> > > https://issues.apache.org/jira/browse/HTTPCLIENT-1176
> > > 
> > > It should be possible to come with a more efficient solution by using a custom
HttpRequestRetryHandler in additional to a custom RoutePlanner.
> > > 
> > > Hope this helps
> > > 
> > > Oleg
> > > 
> > > 
> > > 
> > > ------------------------------------------------------------------
> > > --
> > > - To unsubscribe, e-mail: 
> > > httpclient-users-unsubscribe@hc.apache.org
> > > For additional commands, e-mail: 
> > > httpclient-users-help@hc.apache.org
> > > 
> > > 
> > > ------------------------------------------------------------------
> > > --
> > > - To unsubscribe, e-mail: 
> > > httpclient-users-unsubscribe@hc.apache.org
> > > For additional commands, e-mail: 
> > > httpclient-users-help@hc.apache.org
> > > 
> > 
> > 
> > 
> > --------------------------------------------------------------------
> > - To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> > For additional commands, e-mail: httpclient-users-help@hc.apache.org
> > 
> > 
> > --------------------------------------------------------------------
> > - To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> > For additional commands, e-mail: httpclient-users-help@hc.apache.org
> > 
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
> 



---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org

Mime
View raw message