hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kevin Peterson <kpeter...@biz360.com>
Subject Re: Persistent storage on EC2
Date Thu, 28 May 2009 17:45:54 GMT
On Tue, May 26, 2009 at 7:50 PM, Malcolm Matalka <
mmatalka@millennialmedia.com> wrote:

> I'm using EBS volumes to have a persistent HDFS on EC2.  Do I need to keep
> the master updated on how to map the internal IPs, which change as I
> understand, to a known set of host names so it knows where the blocks are
> located each time I bring a cluster up?  If so, is keeping a mapping up to
> date in /etc/hosts sufficient?

I can't answer your first question of whether it's necessary. The namenode
might be able to figure it out when the DNs report their blocks.

Our staging cluster uses the setup you describe, with /etc/hosts pushed out
to all the machines, and the EBS volumes always mounted on the same
hostname. This works great.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message