hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From yinghua hu <yinghua...@gmail.com>
Subject Re: error running pi program
Date Sat, 10 Nov 2012 18:30:02 GMT
Thanks Steve,

Your suggestion is very helpful. But I only have terminal access to the EC2
NN. To run internet browser there will be a lot of more work.

Since my cluster will be shut down at night and weekend. I will prefer to
have an IP that does not change. How do you think of using EC2 Elastic IP?

On Sat, Nov 10, 2012 at 2:38 AM, Steve Loughran <stevel@hortonworks.com>wrote:

>
>
> On 10 November 2012 00:48, yinghua hu <yinghua.hu@gmail.com> wrote:
>
>> Hi, Ted and Andy
>>
>> I tried both internal and external hostnames. They both worked. But I
>> will elect external host name since this way I can browsing the hdfs file
>> structure and other information from a computer not in the cluster.
>>
>> Thanks!
>>
>
> I wouldn't recommend this:
>
> If you hook up your DNs to the NN via the external addresses, Amazon will
> bill you for external traffic, which could be an unpleasant surprise at the
> end of the month.
>
> If you deploy using Apache Whirr then it gives you a command to run an SSH
> proxy.
>
> I've filed a JIRA for the webhdfs feature in Hadoop 1.0.3+ -which offers a
> way to get at the filesystem via HTTP to not have the NN issue 302
> redirects to the specific DN, but instead act as a proxy; there is a
> cluster proxy service (with the same API) that can do this, but for small
> EC2 clusters using the NN directly would be easier. What I haven't done is
> implemented it -if anyone else wants to I'll gladly review the code (and
> its tests)
>
>
> -Steve
>
>


-- 
Regards,

Yinghua

Mime
View raw message