hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@apache.org>
Subject Re: Suspend or scale back hadoop instance
Date Tue, 19 May 2009 14:23:12 GMT
John Clarke wrote:
> Hi,
> I am working on a project that is suited to Hadoop and so want to create a
> small cluster (only 5 machines!) on our servers. The servers are however
> used during the day and (mostly) idle at night.
> So, I want Hadoop to run at full throttle at night and either scale back or
> suspend itself during certain times.

You could add/remove new task trackers on idle systems, but
* you don't want to take away datanodes, as there's a risk that data 
will become unavailable.
* there's nothing in the scheduler to warn that machines will go away at 
a certain time
If you only want to run the cluster at night, I'd just configure the 
entire cluster to go up and down

View raw message