hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Kalnichevski <ol...@apache.org>
Subject Re: [httpclient] Tunnelling non-HTTP protocols through ProxyClient
Date Thu, 12 May 2005 16:43:42 GMT
I apologize for once again shooting e-mail without having properly
proof-read it. I meant to say "If all you want is to download FTP
resources via an HTTP <strong>proxy</strong>..."

Oleg


On Thu, 2005-05-12 at 18:38 +0200, Oleg Kalnichevski wrote:
> Bindul,
> 
> If all you want to download FTP resources via an HTTP server, this is
> actually quite easy and involves no HTTP tunneling and no black magic.
> Essentially this is what all browsers do, I assume
> 
> HttpClient client = new HttpClient();
> client.getHostConfiguration().setProxy("localhost", 8888);
> Protocol.registerProtocol("ftp", 
>         new Protocol("ftp", new DefaultProtocolSocketFactory(), 21));
> client.getState().setProxyCredentials(
>         new AuthScope("localhost", 8888),
>         new UsernamePasswordCredentials("squid", "squid"));
> GetMethod httpget = new GetMethod("ftp://ftp.whatever.com/");
> try {
>     client.executeMethod(httpget);
>     System.out.println(httpget.getStatusLine());
>     System.out.println(httpget.getResponseBodyAsString());
> } finally {
>     httpget.releaseConnection();
> }
> 
> Besides, consider installing Squid proxy locally on your development
> system and debug your application against it first. Once you are
> reasonably sure your application performs as expected test it against
> your corporate IIS. I do have to say, tough, Squid is horrendously
> difficult to configure properly, but once you get your head wrapped
> around it it is amazingly flexible. I use this approach to debug
> HttpClient
> 
> Hope this helps,
> 
> Oleg
> 
> On Thu, 2005-05-12 at 20:19 +0530, Bindul Bhowmik (GMail) wrote:
> > Oleg,
> > 
> > My aim is to browse and download files over FTP (via an HTTP proxy). I
> > might need to support other proxies later (SFTP for instance), but as
> > I said, that comes later.
> > 
> > For one thing, the proxy I am using does not support FTP proxying. I
> > got that confirmed from our IS guys.
> > 
> > Hope this gives you some info to help me!
> > 
> > - Bindul
> > 
> > On 5/12/05, Oleg Kalnichevski <olegk@apache.org> wrote:
> > > Silly spellcheck operator (me).
> > > 
> > > archive = achieve
> > > 
> > > Sorry about my dismal spelling
> > > 
> > > Oleg
> > > 
> > > On Thu, May 12, 2005 at 04:32:33PM +0200, Oleg Kalnichevski wrote:
> > > > Bindul,
> > > >
> > > > First off, what is it that you are trying to archive: download a file
> > > > hosted on an FTP server via an HTTP proxy or tunnel arbitrary protocols
> > > > via an HTTP proxy? HTTP proxies generally support two modes of
> > > > operation: (1) forwarding requests on behalf of a client using a native
> > > > protocol (proxy must natively support protocols in question) (2)
> > > > tunneling arbitrary (often encrypted) data (the proxy does not have to
> > > > support the protocol used). As it seems the MS proxy you are using has
> > > > been configured to disallow tunneling on all ports other than 443 and
> > > > 8443 for security reasons. At the same time the server can access FTP
> > > > resources in the delegation mode. Depending on what you are trying to
> > > > achieve I can recommend further actions to be taken
> > > >
> > > > Oleg
> > > >
> > > >
> > > > On Thu, May 12, 2005 at 07:37:12PM +0530, Bindul Bhowmik (GMail) wrote:
> > > > > Oleg,
> > > > >
> > > > > I still am not able to figure out the problem. The proxy server we
use
> > > > > is a Microsoft ISA server, which requires NTLM authentication.
> > > > >
> > > > > Could you please post the test code you have? Here is the one I am
> > > > > using to get the socket:
> > > > >
> > > > > <code_snip>
> > > > >     private Socket getSocket(String host, int port) throws IOException
{
> > > > >
> > > > >             HostConfiguration hostConfiguration = new HostConfiguration();
> > > > >             ProxyClient proxyClient = new ProxyClient();
> > > > >
> > > > >             // Proxy information
> > > > >             hostConfiguration.setProxy("proxyserver", 9999);
> > > > >             NTCredentials credentials = new NTCredentials("user",
"password",
> > > > > "proxyserver", "domain");
> > > > >
> > > > >             proxyClient.getState().setProxyCredentials(new
> > > > > AuthScope("proxyserver", AuthScope.ANY_PORT, AuthScope.ANY_SCHEME),
> > > > > credentials);
> > > > >
> > > > >             Protocol ftpProtocol = new Protocol("ftp", new
> > > > > DefaultProtocolSocketFactory(), 21);
> > > > >             hostConfiguration.setHost(host, port, ftpProtocol);
> > > > >
> > > > >             proxyClient.setHostConfiguration(hostConfiguration);
> > > > >
> > > > >
> > > > >             ProxyClient.ConnectResponse connectResponse = proxyClient.connect();
> > > > >
> > > > >             if (connectResponse.getSocket() == null) {
> > > > >                     throw new IOException("Could not connect through
proxy");
> > > > >             }
> > > > >
> > > > >             return connectResponse.getSocket();
> > > > >     }
> > > > > </code_snip>
> > > > >
> > > > > Bindul
> > > > >
> > > > > On 5/12/05, Oleg Kalnichevski <olegk@apache.org> wrote:
> > > > > > Bindul,
> > > > > >
> > > > > > I tested ProxyClient with Squid 2.5 STABLE9 and it worked perfectly
well
> > > > > > for me:
> > > > > >
> > > > > > [DEBUG] header - ->> "CONNECT ftp.bluewin.ch:21 HTTP/1.1"
> > > > > > [DEBUG] header - ->> "User-Agent: Jakarta Commons-HttpClient/3.0-rc2
> > > > > > [\r][\n]"
> > > > > > [DEBUG] header - ->> "Host: ftp.bluewin.ch:21[\r][\n]"
> > > > > > [DEBUG] header - ->> "Proxy-Connection: Keep-Alive[\r][\n]"
> > > > > > [DEBUG] header - ->> "[\r][\n]"
> > > > > > [DEBUG] header - -<< "HTTP/1.0 407 Proxy Authentication
Required
> > > > > > [\r][\n]"
> > > > > > [DEBUG] header - -<< "Server: squid/2.5.STABLE9[\r][\n]"
> > > > > > [DEBUG] header - -<< "Mime-Version: 1.0[\r][\n]"
> > > > > > [DEBUG] header - -<< "Date: Wed, 11 May 2005 20:22:56
GMT[\r][\n]"
> > > > > > [DEBUG] header - -<< "Content-Type: text/html[\r][\n]"
> > > > > > [DEBUG] header - -<< "Content-Length: 1303[\r][\n]"
> > > > > > [DEBUG] header - -<< "Expires: Wed, 11 May 2005 20:22:56
GMT[\r][\n]"
> > > > > > [DEBUG] header - -<< "X-Squid-Error: ERR_CACHE_ACCESS_DENIED
0[\r][\n]"
> > > > > > [DEBUG] header - -<< "Proxy-Authenticate: Basic realm="squid"[\r][\n]"
> > > > > > [DEBUG] header - -<< "X-Cache: MISS from localhost.localdomain[\r][\n]"
> > > > > > [DEBUG] header - -<< "Proxy-Connection: keep-alive[\r][\n]"
> > > > > > [DEBUG] header - ->> "CONNECT ftp.bluewin.ch:21 HTTP/1.0"
> > > > > > [DEBUG] header - ->> "User-Agent: Jakarta Commons-HttpClient/3.0-rc2
> > > > > > [\r][\n]"
> > > > > > [DEBUG] header - ->> "Proxy-Authorization: Basic c3F1aWQ6c3F1aWQ=
> > > > > > [\r][\n]"
> > > > > > [DEBUG] header - ->> "Host: ftp.bluewin.ch:21[\r][\n]"
> > > > > > [DEBUG] header - ->> "Proxy-Connection: Keep-Alive[\r][\n]"
> > > > > > [DEBUG] header - ->> "[\r][\n]"
> > > > > > [DEBUG] header - -<< "HTTP/1.0 200 Connection established[\r][\n]"
> > > > > > 220 195.186.6.165 FTP server ready
> > > > > >
> > > > > > Oleg
> > > > > >
> > > > > >
> > > > > > On Wed, 2005-05-11 at 20:05 +0530, Bindul Bhowmik (GMail) wrote:
> > > > > > > Oleg,
> > > > > > >
> > > > > > > I am not sure if I am missing something here. When I connect
to a ftp
> > > > > > > site using the browser, it uses the same proxy and tunnel
(or am I
> > > > > > > wrong?). Is there something else I need to do to go through
the tunnel
> > > > > > > and connect to an ftp site?
> > > > > > >
> > > > > > > If anyone is interested I could send the code I am using
to get the socket.
> > > > > > >
> > > > > > > - Bindul
> > > > > > >
> > > > > > > On 5/10/05, Oleg Kalnichevski <olegk@apache.org>
wrote:
> > > > > > > > Bindul,
> > > > > > > >
> > > > > > > > HTTP CONNECT (aka HTTP tunneling) has been primarily
designed to enable
> > > > > > > > secure (primarily SSL) connections via HTTP proxies,
hence the error
> > > > > > > > message. There's absolutely nothing that prevents
other protocols from
> > > > > > > > being tunneled in the same manner, provided the proxy
is configured to
> > > > > > > > allow outgoing connections to a particular port. My
_guess_ this
> > > > > > > > problem caused by the ISS configuration, rather than
a bug in HttpClient
> > > > > > > > or your code
> > > > > > > >
> > > > > > > > Oleg
> > > > > > > >
> > > > > > > > On Tue, May 10, 2005 at 09:57:58PM +0530, Bindul Bhowmik
(GMail) wrote:
> > > > > > > > > Oleg,
> > > > > > > > >
> > > > > > > > > The port here is 21. I get this from the configuration
of the FTP host
> > > > > > > > > I have to connect to through the HTTP tunnel.
I am not sure where the
> > > > > > > > > SSL port comes in from!
> > > > > > > > >
> > > > > > > > > FYI, the proxy we use is a Microsoft ISA server
requiring NTLM
> > > > > > > > > authentication. And except for
> > > > > > > > > > >               proxyClient.getState().setProxyCredentials(new
AuthScope("proxy",
> > > > > > > > > > > -1), credentials);
> > > > > > > > > the rest of the piece works fine with httpclient-2.0.2
and HTTPClient
> > > > > > > > > instead of ProxyClient, which we use to download
files over HTTP in
> > > > > > > > > the same application.
> > > > > > > > >
> > > > > > > > > I had to move to 3.0rc2 since ProxyClient or
> > > > > > > > > HTTPConnection#getSocket() were not available
in 2.0.2
> > > > > > > > >
> > > > > > > > > - Bindul
> > > > > > > > >
> > > > > > > > > On 5/10/05, Oleg Kalnichevski <olegk@apache.org>
wrote:
> > > > > > > > > > Bindul,
> > > > > > > > > >
> > > > > > > > > > > hostConfiguration.setHost(host, port,
httpClientProtocol);
> > > > > > > > > >
> > > > > > > > > > What is the value of the port parameter?
If it is not 443 are you sure
> > > > > > > > > > the proxy has been configured to allow outgoing
connections to that
> > > > > > > > > > port?
> > > > > > > > > >
> > > > > > > > > > Oleg
> > > > > > > > > >
> > > > > > > > > > On Tue, May 10, 2005 at 08:41:05PM +0530,
Bindul Bhowmik (GMail) wrote:
> > > > > > > > > > > Hi,
> > > > > > > > > > >
> > > > > > > > > > > I am trying to tunnel a FTP stream
over an HTTP Tunnel proxy. I am
> > > > > > > > > > > trying to use the ProxyClient class
of commons-client for that. I am
> > > > > > > > > > > using commons-net as my FTP Client,
and have written an implementation
> > > > > > > > > > > of SocketFactory to be used for FTP
connections over the Proxy.
> > > > > > > > > > >
> > > > > > > > > > > Inside my SocketFactory implementation,
I am getting the socket from
> > > > > > > > > > > ProxyClient.ConnectResponse#getSocket()
class. The code I use is:
> > > > > > > > > > >
> > > > > > > > > > > <code_snip>
> > > > > > > > > > >                 ProxyClient proxyClient
= new ProxyClient();
> > > > > > > > > > >               HostConfiguration hostConfiguration
= proxyClient.getHostConfiguration();
> > > > > > > > > > >               hostConfiguration.setHost(host,
port, httpClientProtocol);
> > > > > > > > > > >               hostConfiguration.setProxy("proxy",
8085);
> > > > > > > > > > >               NTCredentials credentials
= getNTCredentials("domain\\user",
> > > > > > > > > > >                               "password","proxy");
> > > > > > > > > > >
> > > > > > > > > > >               // Set the proxy credentials
> > > > > > > > > > >               proxyClient.getState().setProxyCredentials(new
AuthScope("proxy",
> > > > > > > > > > > -1), credentials);
> > > > > > > > > > >
> > > > > > > > > > >               ProxyClient.ConnectResponse
response = proxyClient.connect();
> > > > > > > > > > >               if (response.getSocket()
== null) {
> > > > > > > > > > >                       throw new IOException("Connection
through proxy could not be opened");
> > > > > > > > > > >               }
> > > > > > > > > > >
> > > > > > > > > > >               return response.getSocket();
> > > > > > > > > > > </code_snip>
> > > > > > > > > > >
> > > > > > > > > > > However, the getSocket() method returns
null, and on setting the log
> > > > > > > > > > > level to FINE, the last response from
the Proxy says:
> > > > > > > > > > >
> > > > > > > > > > > <pre>
> > > > > > > > > > > HTTP/1.1 502 Proxy Error ( The specified
Secure Sockets Layer (SSL)
> > > > > > > > > > > port is not allowed. ISA Server is
not configured to allow SSL
> > > > > > > > > > > requests from this port. Most Web browsers
use port 443 for SSL
> > > > > > > > > > > requests.  )
> > > > > > > > > > > </pre>
> > > > > > > > > > >
> > > > > > > > > > > This problem is related to the ones
discussed in the commons-user
> > > > > > > > > > > threads: http://mail-archives.apache.org/mod_mbox/jakarta-commons-user/200504.mbox/%3c882a6a7e05041101351531c7d1@mail.gmail.com%3e
> > > > > > > > > > > and http://mail-archives.apache.org/mod_mbox/jakarta-commons-user/200504.mbox/%3cOFD1167CC8.7FF51EF6-ONCA256FE8.000AFB2E@qantas.com.au%3e
> > > > > > > > > > > Any help in solving this would be great.
> > > > > > > > > > >
> > > > > > > > > > > Regards,
> > > > > > > > > > > Bindul
> > > > > > > > > > >
> > > > > > > > > > > ---------------------------------------------------------------------
> > > > > > > > > > > To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> > > > > > > > > > > For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > ---------------------------------------------------------------------
> > > > > > > > > To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> > > > > > > > > For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> > > > > > > For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
> > > > > > >
> > > > > >
> > > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> > > > For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
> > > >
> > >
> > 
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
> > 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org


Mime
View raw message