hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Moore" <jamesthepi...@gmail.com>
Subject Re: hadoop on EC2
Date Wed, 04 Jun 2008 06:26:48 GMT
On Tue, Jun 3, 2008 at 5:04 PM, Andreas Kostyrka <andreas@kostyrka.org> wrote:
> Plus to make it even more painful, you cannot easily run it with one simple
> SOCKS server, because you need to defer DNS resolution to the inside the
> cluster, because VM names do resolve to external IPs, while the webservers
> we'd be all interested in reside on the internal 10/8 IPs.

It's easy with foxyproxy.

Run ssh with the -D command:

ssh -D 2324 ec2-75-101-XXX-XX.compute-1.amazonaws.com

Tell FoxyProxy to "use SOCKS proxy for DNS lookups" (tools > foxyproxy
> more > global settings > use SOCKS proxy for DNS lookups)

Configure foxyproxy with rules for when to use local port 2324.  Use
wildcards like http*ec2*internal*.  I put a screenshot on my blog -
http://blog.restphone.com/2008/6/4/foxyproxy-hadoop-and-socks

All the features I cared about worked when set up this way.

(And of course the choice of 2324 isn't special - use any port you like.)
-- 
James Moore | james@restphone.com
Ruby and Ruby on Rails consulting
blog.restphone.com

Mime
View raw message