hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Leonid Fedotov <lfedo...@hortonworks.com>
Subject Re: recommended nodes
Date Tue, 27 Nov 2012 20:03:42 GMT
Jean-Mark,
you are right, SATA may be too slow, especially when you only have 2 drives.
You may get reasonable performance off there SATA drives, but there should be more than 2,
to spread all IO operation on many spindles.
Better to go to SAS.
In any case, try to have network IO and disk IO balanced.

Thank you!

Sincerely,
Leonid Fedotov


On Nov 27, 2012, at 11:52 AM, Jean-Marc Spaggiari wrote:

> Hi Michael,
> 
> so are you recommanding 32Gb per node?
> 
> What about the disks? SATA drives are to slow?
> 
> JM
> 
> 2012/11/26, Michael Segel <michael_segel@hotmail.com>:
>> Uhm, those specs are actually now out of date.
>> 
>> If you're running HBase, or want to also run R on top of Hadoop, you will
>> need to add more memory.
>> Also forget 1GBe got 10GBe,  and w 2 SATA drives, you will be disk i/o bound
>> way too quickly.
>> 
>> 
>> On Nov 26, 2012, at 8:05 AM, Marcos Ortiz <mlortiz@uci.cu> wrote:
>> 
>>> Are you asking about hardware recommendations?
>>> Eric Sammer on his "Hadoop Operations" book, did a great job about this:
>>> For middle size clusters (until 300 nodes):
>>> Processor: A dual quad-core 2.6 Ghz
>>> RAM: 24 GB DDR3
>>> Dual 1 Gb Ethernet NICs
>>> a SAS drive controller
>>> at least two SATA II drives in a JBOD configuration
>>> 
>>> The replication factor depends heavily of the primary use of your
>>> cluster.
>>> 
>>> On 11/26/2012 08:53 AM, David Charle wrote:
>>>> hi
>>>> 
>>>> what's the recommended nodes for NN, hmaster and zk nodes for a larger
>>>> cluster, lets say 50-100+
>>>> 
>>>> also, what would be the ideal replication factor for larger clusters when
>>>> u have 3-4 racks ?
>>>> 
>>>> --
>>>> David
>>>> 10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS
>>>> INFORMATICAS...
>>>> CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION
>>>> 
>>>> http://www.uci.cu
>>>> http://www.facebook.com/universidad.uci
>>>> http://www.flickr.com/photos/universidad_uci
>>> 
>>> --
>>> 
>>> Marcos Luis Ortíz Valmaseda
>>> about.me/marcosortiz <http://about.me/marcosortiz>
>>> @marcosluis2186 <http://twitter.com/marcosluis2186>
>>> 
>>> 
>>> 
>>> 10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS
>>> INFORMATICAS...
>>> CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION
>>> 
>>> http://www.uci.cu
>>> http://www.facebook.com/universidad.uci
>>> http://www.flickr.com/photos/universidad_uci
>> 
>> 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message