hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: Dynamically specify slave nodes
Date Tue, 24 May 2011 11:35:58 GMT

The system is already dynamic right now, unless a configuration limits
that. Slave nodes may enter/exit any time. But you have to keep in
mind that the loss of a DataNode can lead to a lot of network traffic
for rebalancing/re-replicating blocks lost. A TaskTracker loss is not
so expensive to the cluster, OTOH.

Merely starting up a DN/TT individually on a machine on the network
with the right master hosts configuration will see it automatically
enter the NN/JT cluster.

Is this what you are looking to do?

On Tue, May 24, 2011 at 4:13 PM, Francesco De Luca <f.deluca86@gmail.com> wrote:
> Is it possible to dynamically specify the slave nodesĀ  instead to specify
> them statically through the configuration files?
> I want to build a dynamic environment in which each node can enter and exit,
> so i need to specify the new
> entered node as a new slave node for hadoop framework
> Thanks

Harsh J

View raw message