hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun C Murthy <...@hortonworks.com>
Subject Re: Adding Elasticity to Hadoop MapReduce
Date Wed, 14 Sep 2011 21:24:27 GMT

On Sep 14, 2011, at 1:27 PM, Bharath Ravi wrote:

> Hi all,
> I'm a newcomer to Hadoop development, and I'm planning to work on an idea
> that I wanted to run by the dev community.
> My apologies if this is not the right place to post this.
> Amazon has an "Elastic MapReduce" Service (
> http://aws.amazon.com/elasticmapreduce/) that runs on Hadoop.
> The service allows dynamic/runtime changes in resource allocation: more
> specifically, varying the number of
> compute nodes that a job is running on.
> I was wondering if such a facility could be added to the publicly available
> Hadoop MapReduce.

From a long while  you can bring up either DataNodes or TaskTrackers and point them (via config)
to the NameNode/JobTracker and they will be part of the cluster.

Similarly you can just kill the DataNode or TaskTracker and the respective masters will deal
with their loss.


View raw message