hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <cutt...@apache.org>
Subject Re: EC2 Instance manager?
Date Tue, 23 Jan 2007 00:32:31 GMT
Lee wrote:
> I was contemplating working on an EC2 instance manager for use with Hadoop.
> The idea is it would handle instance creation on namenode/jobtracker
> startup, perhaps adding/removing instances during the lifetime of the
> cluster.  Would this be useful?  If so, any ideas/pointers for creating it?

Tom's started some scripts in this direction:


Also, EC2 currently tries to host all instances allocated at one time at 
the same datacenter.  Adding and removing instances is thus likely to 
generate a cluster that spans multiple datacenters, which may not be 
optimal, although, in my experiments, Amazon appears to have a lot of 
inter-datacenter bandwidth, so this may or may not be an issue.


View raw message