hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mohammad Tariq <donta...@gmail.com>
Subject Re: Hardware Selection for Hadoop
Date Mon, 29 Apr 2013 21:16:21 GMT
If I were to start with a 5 node cluster, I would do this :

*Machine 1 : *NN+JT
32GB RAM, 2xQuad Core Proc, 500GB SATA HDD along with a NAS(To make sure
metadata is safe)

*Machine 2 : *SNN*
*
32GB RAM, 2xQuad Core Proc, 500GB SATA HDD

*Machine 3,4,5 : *DN+TT
16GB RAM, 2xQuad Core Proc, 5 x 200GB SATA HDD(JBOD configuation)

I don't think you'll require 64GB RAM and so much of storage just for a
POC(but, it actually depends). You can really kick the ass with 32GB.

NIC(network interface card) is a computer hardware component that connects
a computer to a computer network. It must be reliable to make sure that all
your machines are always connected to the cluster and there is no problem
in the data transfer.

Apart from this ask them to provide you cabinets with good ventilation and
cooling mechanism.

Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com


On Tue, Apr 30, 2013 at 2:17 AM, Raj Hadoop <hadoopraj@yahoo.com> wrote:

> Hi,
>
> In 5 node cluster - you mean
>
> Name Node , Job Tracker , Secondary Name Node all on 1
>         64 GB Ram ( Processor - 2 x Quad cores Intel  , Storage - ? )
>
> Data Trackers and Job Trackers - on 4 machies - each of
>         32 GB Ram ( Processor - 2 x Quad cores Intel  , Storage - ? )
>
> NIC ?
>
> Also - what other details should I provide to my hardware engineer.
>
> The idea is to start with a Web Log Processing proof of concept.
>
> Please advise.
>
>
>   *From:* Patai Sangbutsarakum <Patai.Sangbutsarakum@turn.com>
> *To:* "user@hadoop.apache.org" <user@hadoop.apache.org>
> *Sent:* Monday, April 29, 2013 2:49 PM
> *Subject:* Re: Hardware Selection for Hadoop
>
>  2 x Quad cores Intel
> 2-3 TB x 6 SATA
> 64GB mem
> 2 NICs teaming
>
> my 2 cents
>
>
>  On Apr 29, 2013, at 9:24 AM, Raj Hadoop <hadoopraj@yahoo.com>
>  wrote:
>
>      Hi,
>
> I have to propose some hardware requirements in my company for a Proof of
> Concept with Hadoop. I was reading Hadoop Operations and also saw Cloudera
> Website. But just wanted to know from the group - what is the requirements
> if I have to plan for a 5 node cluster. I dont know at this time, the data
> that need to be processed at this time for the Proof of Concept. So - can
> you suggest something to me?
>
> Regards,
> Raj
>
>
>
>
>

Mime
View raw message