hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Clarke <clarke...@gmail.com>
Subject Re: Suspend or scale back hadoop instance
Date Tue, 19 May 2009 13:47:05 GMT
Hi Piotr,

Thanks for the prompt reply.

If the cron script shuts down Hadoop surely it won't pick up where it left
off when it is restarted?

All the machines will be used during the day so it is not an option to turn
Hadoop off on only some of them.

John




2009/5/19 Piotr Praczyk <piotr.praczyk@gmail.com>

> Hi John
>
> I don't know if there is a Hadoop support for such thing, but You can do
> this easily writing a crontab  script. It could start hadoop at specified
> hour and shut it down ( disable some nodes) at another one.
>
> There can be some problems with HDFS however ( if you disable all the nodes
> holding replicas of some blocks, the files will become inaccessible)
>
> Piotr
>
> 2009/5/19 John Clarke <clarkemjj@gmail.com>
>
> > Hi,
> >
> > I am working on a project that is suited to Hadoop and so want to create
> a
> > small cluster (only 5 machines!) on our servers. The servers are however
> > used during the day and (mostly) idle at night.
> >
> > So, I want Hadoop to run at full throttle at night and either scale back
> or
> > suspend itself during certain times.
> >
> > Is it possible to do this? I've found very little information on it.
> >
> > Thanks for your help!
> > John
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message