hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mikhail Yakshin <greycat.na....@gmail.com>
Subject Re: Hard drives dropping like flies
Date Tue, 21 Jul 2009 15:11:58 GMT
On Tue, Jul 21, 2009 at 9:44 AM, Tim
Nelson<hadoop@enigmasupercomputing.com> wrote:
> I have a question that I think I already know the answer to but I would like
> verification.  I have a demo cluster comprised of two master nodes and eight
> slaves (all 1x1.2 Ghz cpu / 1 Gig Ram / 1x250 Gig Sata 7200 rpm hard
> drives).

[...]

> If this is the problem, do I need to be sure and place my dfs data and
> mapred data directories onto separate physical drives?  If I upgrade to some
> dell servers with dual core and 4-5HD's per node will hadoop take care of
> balancing the load between hard drives (dfs/mapred/scratch storage)?

Looks pretty much like a hardware problem? Do you test your servers
before they go into production? Any power/temperature issues?

We use some kind of a free GPLed hardware testing system/platform that
could be used to test/diagnose such things before servers go live. You
can check it out at http://www.inquisitor.ru/

-- 
WBR, Mikhail Yakshin

Mime
View raw message